AI Pair Programming, Six Months In: What Changed

We introduced AI pair programming across the team roughly six months ago, alongside the AI code review process and QA changes we've written about separately. Enough time has passed to talk about what actually changed, rather than what we expected to change going in - and the honest answer is more nuanced than "we ship faster now."

What Genuinely Got Faster

Boilerplate and scaffolding work - setting up standard project structure, writing initial test scaffolding, drafting repetitive CRUD logic - dropped meaningfully in time spent. This is the category AI pair programming was always going to help with most, and it did: the kind of code that's necessary but doesn't require much judgment to write correctly. Developers report spending less time on the mechanical first draft of this work and more time reviewing and adjusting it, which is a real shift in where effort goes even when the total time isn't dramatically shorter.

What Didn't Get Faster, and Why That's Fine

Architectural decisions, debugging genuinely novel issues, and anything requiring deep context about a specific client's business logic didn't speed up in any measurable way. This wasn't a disappointment - it's exactly where we expected AI assistance to plateau, since these tasks depend on context and judgment that AI pair programming tools don't have direct access to. We track this distinction specifically because it's easy for a vague sense of "AI made us faster" to obscure that the speedup is concentrated in specific categories of work, not spread evenly across everything.

Where We Had to Actively Correct Course

The most concrete lesson from six months in: early on, we noticed reviewers spending more time than expected re-verifying AI-suggested code that looked correct but relied on subtly outdated assumptions about a library version or an internal convention that had changed. The code wasn't wrong in isolation - it was wrong for our specific, current context. This is part of what drove the QA process changes we made around combined-condition testing, and it's also why our code review process treats AI-suggested code as a draft requiring the same scrutiny as a junior developer's first pass, not less.

How Developer Behavior Actually Shifted

The more interesting change wasn't speed - it was what developers spend their attention on. Reviewing AI-suggested code requires a different kind of attention than writing code from scratch: verifying reasoning rather than generating it. Some developers adapted to this quickly and now review AI suggestions critically as a matter of habit. Others took longer, initially accepting suggestions with less scrutiny than we wanted, which is part of why the human-authorship boundaries we've written about - for authentication, financial logic, and migrations specifically - exist as firm rules rather than case-by-case judgment calls.

What We Measure Now, Six Months In

Rather than a single productivity metric, we track: time spent on boilerplate versus judgment-heavy work, the rate at which AI suggestions require substantial correction during review, and - most importantly - whether bug rates in AI-assisted code differ from human-only code once both go through the same review process. That last number is the one that actually tells us whether the tool is helping or just feels helpful, and it's the one we're most careful not to round up.

What Changes Next

We're not expanding AI's role uniformly going forward. Where the six-month data shows AI assistance genuinely reduces review burden without increasing correction rates, we're comfortable extending it further. Where correction rates stay elevated, we're pulling back rather than pushing through on the assumption that the tooling will improve fast enough to justify the current risk. The goal was never maximum AI usage - it was better use of the team's attention, and that's the number we keep checking against.