What We Won't Let AI Touch in Our Codebase
Where AI drafts and where a human writes the logic directly
We've written before about how Zoraz AI reviews its own code before a human does, and about a security audit that found one bug in AI-drafted code. Both of those posts assumed AI is already deeply embedded in how we build software - which it is. What we haven't laid out directly is where we stop letting it in. That boundary isn't accidental, and it isn't shrinking over time the way some might assume. If anything, using AI more has made us more specific about where it doesn't belong.
Authentication and Access Control Logic
AI can suggest authentication flows, and it can draft boilerplate around them. It does not get to independently write or modify the logic that determines who can access what. This is the category of code where a subtle, plausible-looking mistake has the highest blast radius - a permission check that looks correct but is evaluated in the wrong order, for instance, can silently expose data for months before anyone notices. Every line in this category gets written or directly rewritten by a human, with AI limited to a supporting, suggestion-only role.
Anything Touching Financial Calculations
Pricing logic, discount stacking, payment amounts, refund calculations - anything where a bug produces a wrong dollar figure rather than a visibly broken feature. Wrong financial output is dangerous specifically because it often doesn't look wrong. A broken UI is obvious. An order total that's off by a rounding error in a specific edge case can run in production for a long time before anyone catches it, as we found out firsthand. AI can flag test cases and suggest structure here, but the actual calculation logic gets human authorship and, per our QA process changes, explicit combined-condition testing before it ships.
Data Migration Scripts
A migration script runs once, often against production data, and a mistake isn't easily reversible. AI-generated migration logic gets treated with more suspicion than AI-generated application code generally, because the cost of being wrong is asymmetric - a bad migration can corrupt data in a way that's expensive or impossible to fully undo. We allow AI to draft the scaffolding of a migration script, but the actual transformation logic is human-written and manually verified against a full data backup before it runs anywhere real.
Security-Sensitive Configuration
Firewall rules, API key scoping, environment variable handling, anything that defines what's exposed and to whom. This is a narrow category, but it's one where an AI-suggested "convenience" default has repeatedly turned out to be the wrong default from a security standpoint - broader access scopes than necessary, for instance, because a broader scope is more likely to "just work" during testing. We treat any AI suggestion in this category as a draft to scrutinize, never as something to accept as-is.
What AI Handles Well Instead
None of this is a statement that AI involvement is limited overall - it isn't. Boilerplate, test scaffolding, refactoring within an already-reviewed pattern, documentation, and a meaningful share of standard application logic all go through AI assistance first, with our review process (both the AI pre-review pass and human review) catching what needs catching. The line isn't "AI versus no AI." It's specifically about consequence and reversibility - the categories above share the trait that a mistake is either hard to detect or hard to undo, and that combination is where we keep AI in a supporting role rather than a drafting one.
Why This List Isn't Fixed Forever
We revisit this list periodically, not to expand it reflexively as AI tools improve, but to check whether our reasoning still holds. If AI tooling develops better verification specifically for financial logic or migrations - not just better code generation, but better proof that the generated logic is correct - that would change our comfort level in a specific category. Until then, the boundary stays where the actual risk is.
Frequently Asked Questions
Does Zoraz AI use AI to write authentication or permissions code?
No. AI can suggest structure, but access-control logic is written and reviewed by a human because mistakes here can silently expose data.
Why are financial calculations treated differently from other code?
Wrong financial output often doesn't look visibly broken, unlike a broken UI, so it can run undetected for a long time. That risk profile keeps the actual calculation logic human-authored.
Is AI allowed to write data migration scripts?
AI can draft scaffolding, but the transformation logic itself is human-written and verified against a backup before running, since migration mistakes are often hard to reverse.
Will this list of restricted areas shrink as AI tools improve?
Only where the underlying risk changes — for example, if verification tooling for financial or migration logic improves meaningfully. It isn't expanded or shrunk based on general AI capability alone. Publish Published: Yes Featured blog: (your call) Show on homepage: (your call) Category AI & Engineering Author Name Weboraz Team Tags AI in engineering, code review, security, software development, Zoraz AI SEO SEO Title What We Won't Let AI Touch in Our Codebase | Zoraz AI Meta Description AI drafts a meaningful share of our code. Here's exactly where we draw the line — authentication, financial logic, migrations, and security config — and why. Focus Keyword AI code restrictions Secondary Keywords AI in software development, AI code review boundaries, secure coding practices Canonical URL /blog/what-we-wont-let-ai-touch-in-our-codebase Meta Robots index, follow Social Sharing OG Title What We Won't Let AI Touch in Our Codebase OG Image (needs upload) OG Description AI writes a meaningful share of our code. It doesn't touch authentication, financial logic, migrations, or security config directly. Here's why. Twitter Title What We Won't Let AI Touch in Our Codebase Twitter Image (needs upload) Twitter Description More AI in our workflow made us more specific about where it doesn't belong, not less. Here's the actual boundary. Post 2 Title Testing in Production vs Testing Before Production: Our Approach Slug /blog/testing-in-production-vs-testing-before-production-our-approach Excerpt "Testing in production" sounds like a shortcut or a joke, depending who you ask. In practice, it's a deliberate part of how we validate systems — used specifically, not as a substitute for testing before launch. Main Content "We test in production" is usually said as a joke, an admission that something shipped without enough validation. Said seriously, it means something different — and more useful. There are things you genuinely cannot verify before real traffic hits a system, and pretending otherwise just means finding out the hard way, unplanned, instead of on purpose. Here's how we actually think about the split. What Has to Be Validated Before Production Anything with a clear correct answer gets tested before launch, without exception. Core business logic, calculation accuracy, security boundaries, data integrity constraints — these have a right answer independent of real traffic, and there's no reason to wait for production to check them. Unit tests, integration tests, and staging environments with realistic data handle this category, and we don't treat "we'll catch it in prod" as an acceptable substitute for it. What Only Production Can Actually Tell You Some things don't have a knowable right answer until real usage exists. How a system behaves under genuinely unpredictable concurrent load, how caching performs against real user access patterns instead of synthetic ones, how a recommendation or ranking algorithm performs against actual user behavior rather than modeled behavior — staging can approximate these, but only production traffic tells you the real answer. Pretending staging fully substitutes for this is how systems pass every test and then behave unexpectedly on day one anyway. How We Actually Test in Production, Safely The phrase implies more recklessness than the practice actually involves. When we do validate something against real production conditions, it's through controlled mechanisms specifically built for it: feature flags that let us enable new logic for a small percentage of real traffic before a full rollout, canary deployments that route a fraction of requests to a new version while monitoring closely, and shadow traffic — where a new system processes a copy of real requests without its output affecting real users, purely to compare behavior. None of this is "ship it and see what breaks." It's structured exposure to real conditions with a fast, deliberate rollback path if something looks wrong. The Line We Don't Cross We don't use production testing for anything where a failure has an unrecoverable or high-cost consequence for a real user — financial transactions, data-altering operations, anything security-related. That category gets exhaustively tested before production, full stop, because the cost of being wrong in front of a real user isn't acceptable even at small scale. Production testing is reserved for things where a wrong result is observable, contained, and reversible. Why We Don't Treat This as an Either/Or The framing of "testing in production versus testing before production" implies you pick one philosophy and commit. We don't. Both exist in the same pipeline, applied to different categories of risk — pre-production testing for anything with a checkable right answer, structured production testing for anything that genuinely can't be known until real conditions exist. Treating them as competing philosophies usually means either over-relying on staging environments that never quite match reality, or under-testing before launch and calling it agile. What This Looks Like on a Real Project On a recent client project involving a personalized content feed, we validated data accuracy, security, and core logic entirely pre-production — that had a right answer and staging could confirm it. Ranking quality, however, only became measurable once real user interaction data existed, so we rolled the ranking logic out to five percent of traffic, watched engagement metrics against the existing system for two weeks, and expanded gradually as the numbers held up. Neither phase substituted for the other — each answered a question the other couldn't. Featured Image Alt Text Diagram showing a feature flag rollout gradually expanding from a small percentage of production traffic Image Caption Controlled exposure to real traffic, with a fast rollback path if something looks wrong Video Embed URL (none) Gallery Images (none) FAQ Q: Is "testing in production" the same as skipping pre-launch testing? A: No. Anything with a checkable right answer — core logic, security, data integrity — is fully tested before launch. Production testing is reserved for things that genuinely can't be validated any other way. Q: What can only be learned by testing in real production conditions? A: Behavior under real concurrent load, caching performance against actual access patterns, and how ranking or recommendation logic performs against real user behavior rather than modeled behavior. Q: How do you test in production without risking real users? A: Through controlled mechanisms — feature flags exposing new logic to a small percentage of traffic, canary deployments, and shadow traffic that compares output without affecting real users. Q: What do you never test directly in production? A: Anything with an unrecoverable or high-cost failure mode for a real user — financial transactions, data-altering operations, and security-related logic are fully validated beforehand. Publish Published: Yes Featured blog: (your call) Show on homepage: (your call) Category Engineering Process Author Name Weboraz Team Tags software testing, production deployment, feature flags, engineering process, QA SEO SEO Title Testing in Production vs Testing Before Production: Our Approach | Zoraz AI Meta Description "Testing in production" isn't a shortcut when it's done right. Here's how we decide what gets validated before launch and what genuinely needs real traffic to test. Focus Keyword testing in production Secondary Keywords pre-production testing, feature flag rollout, canary deployment Canonical URL /blog/testing-in-production-vs-testing-before-production-our-approach Meta Robots index, follow Social Sharing OG Title Testing in Production vs Testing Before Production: Our Approach OG Image (needs upload) OG Description Some things have a right answer you can test before launch. Some things only real traffic can tell you. Here's how we split the two. Twitter Title Testing in Production vs Testing Before Production: Our Approach Twitter Image (needs upload) Twitter Description "We test in production" usually means something got skipped. Done deliberately, it means something else entirely. Here's how we actually use it. Post 3 Title How We Chose Our Tech Stack (and When We'll Break Our Own Rules) Slug /blog/how-we-chose-our-tech-stack-and-when-well-break-our-own-rules Excerpt A default tech stack exists to save time on decisions that don't need to be re-litigated every project. Here's how we settled on ours, and the specific conditions under which we override it anyway. Main Content Having a default tech stack is less about believing it's the objectively best option in every case, and more about not re-deciding foundational architecture from scratch on every single project. A default is a starting position, not a rule without exceptions — and the exceptions matter as much as the default itself. What We Actually Optimized For Our default stack decisions weren't made by chasing what's newest or most discussed. We weighed a specific, narrower set of factors: how many engineers can competently maintain the codebase later, not just build it initially; how mature and stable the ecosystem is for the kind of applications we build most often; how well it supports the maintainability our clients need, since most of our relationships extend well past initial launch into ongoing support; and how predictable its hosting and scaling costs are at the traffic levels our typical client actually reaches, not at hypothetical massive scale. None of these factors reward chasing the newest framework the moment it appears. Stability and maintainability compound in value over a multi-year client relationship in a way that a marginally faster benchmark doesn't. Why a Default Stack Matters More Than People Assume A consistent default means our team can move faster on new projects because the foundational decisions aren't being re-litigated every time. It means a developer coming onto a project mid-stream, or picking up support work on something built a year ago, isn't learning an unfamiliar stack on top of learning the specific codebase. And it means our estimation process — which depends on genuinely understanding the work involved — is more accurate, because we're estimating against a stack the team already knows deeply rather than one they're learning as they go. Where We Break From It Anyway The default doesn't hold when a project has a specific, well-justified reason to diverge. If a client's team will maintain the codebase after handoff and already has deep expertise in a different stack, we build in what they can actually support long-term — a technically excellent solution nobody on the client side can maintain isn't actually a good outcome for them. If a project has a specific technical requirement our default stack handles poorly — certain real-time or high-concurrency workloads, for instance — we use what the requirement actually calls for. And if we're integrating tightly with an existing system built on a different stack, fighting that reality to preserve our own default usually costs more than it saves. What We Don't Treat as a Valid Reason to Deviate We don't switch stacks because a new framework is generating buzz, because a developer personally prefers a different tool, or because a client has heard a specific technology is "the modern choice" without a concrete reason attached to their situation. These are the reasons stacks tend to fragment across an engineering team for no real benefit, and we've seen what that fragmentation costs in maintainability later — a codebase that's a patchwork of individual technology preferences is harder to hand off, harder to hire against, and harder to support consistently. How the Decision Actually Gets Made When a deviation is proposed, we ask the same question every time: does divergence solve a specific, real requirement of this project, or does it solve a preference. The first gets a serious evaluation of trade-offs. The second gets weighed against the real cost of stack fragmentation, and usually loses. Revisiting the Default Itself The default stack isn't static either — it gets reviewed periodically against the same criteria we used to choose it originally: maintainability, ecosystem maturity, team capability, and cost predictability. When something changes enough on those dimensions to justify updating the default, we do. What we avoid is treating every new tool that appears as a reason to reconsider — the bar for changing the default is the same bar we use for justifying an exception, applied to the whole team rather than one project. Featured Image Alt Text Diagram comparing default tech stack criteria against project-specific exceptions Image Caption Maintainability and team capability weighed against buzz-driven technology choices Video Embed URL (none) Gallery Images (none) FAQ Q: What criteria determine your default tech stack? A: Long-term maintainability, ecosystem maturity, team capability to support it, and predictable hosting and scaling costs at realistic traffic levels — not how new or trending a technology is. Q: When do you deviate from your default stack? A: When a client's own team needs to maintain the code and already has different expertise, when a project has a specific technical requirement the default handles poorly, or when integrating tightly with an existing different-stack system. Q: What's not a good enough reason to switch technologies on a project? A: A new framework generating buzz, individual developer preference, or a client hearing a technology is "the modern choice" without a concrete reason tied to their actual situation. Q: Does the default stack itself ever change? A: Yes, periodically, reviewed against the same criteria used to choose it originally — but not simply because a new tool has appeared. Publish Published: Yes Featured blog: (your call) Show on homepage: (your call) Category Engineering Process Author Name Weboraz Team Tags tech stack, software architecture, engineering decisions, maintainability, technology selection SEO SEO Title How We Chose Our Tech Stack (and When We'll Break Our Own Rules) | Zoraz AI Meta Description A default tech stack exists to avoid re-deciding architecture every project. Here's what we actually optimized for, and the specific conditions where we deviate from it. Focus Keyword tech stack decision framework Secondary Keywords choosing a tech stack, software architecture decisions, engineering team standards Canonical URL /blog/how-we-chose-our-tech-stack-and-when-well-break-our-own-rules Meta Robots index, follow Social Sharing OG Title How We Chose Our Tech Stack (and When We'll Break Our Own Rules) OG Image (needs upload) OG Description A default stack isn't about chasing what's newest. Here's what we actually optimized for, and the specific conditions under which we deviate from it. Twitter Title How We Chose Our Tech Stack (and When We'll Break Our Own Rules) Twitter Image (needs upload) Twitter Description Our default stack isn't a rule without exceptions. Here's what earns an exception, and what doesn't. Post 4 Title AI Pair Programming, Six Months In: What Changed Slug /blog/ai-pair-programming-six-months-in-what-changed Excerpt Six months of AI-assisted development changed less about our output and more about how our team spends its attention. Here's what actually shifted, in specific terms. Main Content We introduced AI pair programming across the team roughly six months ago, alongside the AI code review process and QA changes we've written about separately. Enough time has passed to talk about what actually changed, rather than what we expected to change going in — and the honest answer is more nuanced than "we ship faster now." What Genuinely Got Faster Boilerplate and scaffolding work — setting up standard project structure, writing initial test scaffolding, drafting repetitive CRUD logic — dropped meaningfully in time spent. This is the category AI pair programming was always going to help with most, and it did: the kind of code that's necessary but doesn't require much judgment to write correctly. Developers report spending less time on the mechanical first draft of this work and more time reviewing and adjusting it, which is a real shift in where effort goes even when the total time isn't dramatically shorter. What Didn't Get Faster, and Why That's Fine Architectural decisions, debugging genuinely novel issues, and anything requiring deep context about a specific client's business logic didn't speed up in any measurable way. This wasn't a disappointment — it's exactly where we expected AI assistance to plateau, since these tasks depend on context and judgment that AI pair programming tools don't have direct access to. We track this distinction specifically because it's easy for a vague sense of "AI made us faster" to obscure that the speedup is concentrated in specific categories of work, not spread evenly across everything. Where We Had to Actively Correct Course The most concrete lesson from six months in: early on, we noticed reviewers spending more time than expected re-verifying AI-suggested code that looked correct but relied on subtly outdated assumptions about a library version or an internal convention that had changed. The code wasn't wrong in isolation — it was wrong for our specific, current context. This is part of what drove the QA process changes we made around combined-condition testing, and it's also why our code review process treats AI-suggested code as a draft requiring the same scrutiny as a junior developer's first pass, not less. How Developer Behavior Actually Shifted The more interesting change wasn't speed — it was what developers spend their attention on. Reviewing AI-suggested code requires a different kind of attention than writing code from scratch: verifying reasoning rather than generating it. Some developers adapted to this quickly and now review AI suggestions critically as a matter of habit. Others took longer, initially accepting suggestions with less scrutiny than we wanted, which is part of why the human-authorship boundaries we've written about — for authentication, financial logic, and migrations specifically — exist as firm rules rather than case-by-case judgment calls. What We Measure Now, Six Months In Rather than a single productivity metric, we track: time spent on boilerplate versus judgment-heavy work, the rate at which AI suggestions require substantial correction during review, and — most importantly — whether bug rates in AI-assisted code differ from human-only code once both go through the same review process. That last number is the one that actually tells us whether the tool is helping or just feels helpful, and it's the one we're most careful not to round up. What Changes Next We're not expanding AI's role uniformly going forward. Where the six-month data shows AI assistance genuinely reduces review burden without increasing correction rates, we're comfortable extending it further. Where correction rates stay elevated, we're pulling back rather than pushing through on the assumption that the tooling will improve fast enough to justify the current risk. The goal was never maximum AI usage — it was better use of the team's attention, and that's the number we keep checking against. Featured Image Alt Text Chart comparing developer time spent on boilerplate versus judgment-heavy work before and after AI pair programming adoption Image Caption Where AI pair programming actually shifted developer time, and where it didn't Video Embed URL (none) Gallery Images (none) FAQ Q: What specifically got faster after adopting AI pair programming? A: Boilerplate and scaffolding work — standard project structure, initial test scaffolding, repetitive logic — where correctness doesn't require deep judgment. Q: Did AI pair programming speed up architectural decisions or debugging? A: No, and that was expected. Those tasks depend on context and judgment that AI tools don't have direct access to. Q: What problem did you have to actively correct for? A: AI-suggested code that looked correct but relied on outdated assumptions about library versions or internal conventions specific to the current codebase — it fed directly into process changes around combined-condition testing. Q: How do you measure whether AI pair programming is actually helping, six months in? A: By tracking time spent on boilerplate versus judgment work, how often AI suggestions need substantial correction, and whether bug rates differ between AI-assisted and human-only code after the same review process. Publish Published: Yes Featured blog: (your call) Show on homepage: (your call) Category AI & Engineering Author Name Weboraz Team Tags AI pair programming, developer productivity, AI in engineering, software development, Zoraz AI SEO SEO Title AI Pair Programming, Six Months In: What Changed | Zoraz AI Meta Description Six months of AI pair programming didn't speed up everything evenly. Here's exactly what changed, what didn't, and what we had to actively correct for. Focus Keyword AI pair programming Secondary Keywords AI in software development, developer productivity AI, AI code assistance Canonical URL /blog/ai-pair-programming-six-months-in-what-changed Meta Robots index, follow Social Sharing OG Title AI Pair Programming, Six Months In: What Changed OG Image (needs upload) OG Description The speedup wasn't even across everything. Here's exactly what changed after six months of AI pair programming, and what we had to correct for. Twitter Title AI Pair Programming, Six Months In: What Changed Twitter Image (needs upload) Twitter Description Boilerplate got faster. Architecture didn't. Here's the honest, specific breakdown of what six months of AI pair programming actually changed. Updated memory Updated memory All four posts are ready above. Let me know when you want to move into the client-based blog format, or if you'd like more topics in this style.
Ready to build something like this?
Let’s talk about what AI-accelerated, human-validated development can do for your business.
Start Your Project