What an AI Code Health Case Study Can Show

What an AI Code Health Case Study Can Show

A pull request can now arrive with 800 lines of plausible code before its author has finished describing the task. That changes the review load, not just the typing speed. This AI code health case study examines what to measure when a team adopts coding assistants and wants an answer more useful than “output went up.”

The figures below describe an illustrative, reproducible team scenario. They are not a customer result or an industry benchmark. The point is the measurement method: establish a baseline, compare similar periods, inspect where the change landed, and state what the data cannot prove.

The case: faster pull requests, slower confidence

A 12-engineer team maintains three TypeScript services and a shared internal package. The services have regular production traffic, but most work is feature development rather than incident response. During an eight-week baseline period, the team merged 248 pull requests. In the next eight weeks, after making AI coding assistants available to everyone, it merged 329.

That is a 33% increase in merged pull requests. It is not, by itself, a 33% increase in delivered value. Pull request count can rise because work was split more finely, because generated code is more verbose, or because review standards changed. The team therefore recorded several signals alongside throughput: pull request size, review rounds, test changes, rework, files touched repeatedly, and ownership concentration.

The first result looked good. Median time from first commit to merge fell from 2.8 days to 2.1 days. Small changes such as API client updates, validation rules, and test fixture work moved faster. Engineers spent less time looking up library syntax and writing repetitive adapters.

The second result was less comfortable. Median pull request size rose from 186 changed lines to 264. Average review rounds increased from 1.6 to 2.1. Reviewers were not finding more syntax errors. They were asking more questions about boundary conditions, error handling, and whether a new abstraction belonged in a shared package.

This distinction matters. AI can reduce the cost of producing a first draft while increasing the amount of code a reviewer must understand before approving it.

What this AI code health case study measured

Code health is not a single score. A maintainable codebase can still contain old modules, uneven test coverage, and high-churn files. The useful question is whether change is becoming easier or harder for the people who have to make the next change.

The team used five measurements. None was treated as a verdict on its own.

First, it measured delivery flow: time to merge, pull requests merged, and pull request size. These show whether work is moving, but not whether it will remain easy to change.

Second, it measured review friction: number of review rounds, time waiting for review, and comments requesting a redesign rather than a correction. A rise in review rounds is not automatically bad. It can mean reviewers are catching problems earlier. It becomes a concern when it persists while pull request size and rework also rise.

Third, it measured near-term rework. A change was marked as rework when the same feature area needed a corrective pull request within 14 days. The baseline rate was 9%. In the AI-assisted period, it rose to 15%.

Fourth, it tracked hotspot churn. The team identified files changed by at least three separate pull requests in a two-week window, then checked whether the changes came from unrelated tasks. Repeated edits to a payment adapter may be expected during a migration. Repeated edits to a shared authorization helper often mean the interface is unclear or its behavior is not well covered by tests.

Fifth, it tracked ownership concentration. For each directory, the team looked at who reviewed and changed it over the prior 90 days. The goal was not to assign blame or enforce a single owner. It was to find areas where one person remained the only reviewer able to explain a generated change.

Where the maintenance cost appeared

The additional rework did not spread evenly across the repository. It clustered at interfaces: serialization boundaries, permission checks, data migrations, and code shared by more than one service. These are places where locally correct code can still violate assumptions held elsewhere.

For example, an assistant generated a mapper that accepted an optional field and supplied a default. The code compiled, the focused unit test passed, and the pull request was merged quickly. A downstream consumer used the distinction between an omitted value and an explicit default. The corrective change was small, but finding the contract took longer than writing the original code.

The issue was not that the assistant invented an unusual pattern. It used a common pattern without access to the team’s unstated rule. That is the recurring risk: generated code tends to be strongest when the expected behavior is visible in the current file, local tests, and established types. It is weaker when correctness depends on history, operational behavior, or a convention that exists only in somebody’s memory.

The team also found that test volume was a poor proxy for confidence. Test files changed in 61% of AI-assisted pull requests, compared with 49% in the baseline. Yet several added tests repeated the implementation instead of describing behavior at the boundary. More tests did not automatically mean better checks.

Production incident count did not change during the 16-week observation window. That was useful, but not decisive. The sample was short, incident volume was low, and many maintenance costs appear months after a pattern enters a shared module. The team did not claim that AI caused the increase in rework. It only found that the timing, location, and review data justified changing its workflow.

The controls that improved the signal

The team did not ban AI tools. That would have removed a real speed gain from low-risk work. Instead, it made the use of generated code easier to inspect.

Pull requests that introduced a new shared abstraction had to state the behavior being preserved and name the callers affected. This added a few minutes for the author, but it gave reviewers a concrete contract to test. For changes to permissions, migrations, and public APIs, the team asked for one test that would fail if a consumer relied on the old behavior.

It also set a practical pull request expectation: if generated code made a change much larger than the task description, split the mechanical edits from the design decision. Reviewers could then assess the contract before spending time on repetitive changes.

Finally, the team reviewed hotspots every two weeks. A dashboard such as Wizhut .Code can bring together repository trends, maintainability signals, and ownership patterns, but it does not establish cause on its own. The useful part happens when engineers inspect the affected files and decide whether the trend reflects planned work, a weak interface, or a review gap.

How to run this measurement in your repository

Start with a period before any policy change, ideally six to eight weeks. Use the same repositories, exclude large migrations when possible, and write down releases or staffing changes that could distort the comparison. Then collect another period after adoption. Do not compare one unusually busy sprint with one quiet sprint.

Keep the first pass narrow. Measure merge time, pull request size, review rounds, near-term rework, and hotspot churn. Add ownership concentration if your team has directories that only one or two people can safely modify. Numbers should lead you to a pull request or file you can inspect, not end the investigation.

You also need a definition of AI use. Self-reported labels are imperfect, but they are better than pretending attribution is available from Git history. A simple pull request checkbox can distinguish no assistance, assistance for exploration or tests, and assistance for substantive production code. Treat it as context, not surveillance.

The outcome may differ by codebase. A mature, strongly typed service with clear tests may see faster delivery and no meaningful health change. A fast-moving monorepo with informal boundaries may accumulate review and rework cost quickly. The difference is not whether the team uses AI. It is whether the repository makes its contracts visible enough for both authors and reviewers to check them.

Run the measurement before setting a broad policy. Your next useful decision may be as small as requiring one boundary test for generated changes in shared code. That is easier to adopt, easier to verify, and more likely to protect the speed your team wanted in the first place.

.Code

Insights for your code and the team behind it. .Code reads your GitHub and GitLab organizations, every repository's whole history, and says in plain words how the work is progressing, how maintainable the code is, who it rests on, and how the code AI writes holds up.