KaaS Newsletter

KaaS Newsletterhttps://kaas.team/newsletter Research on what AI changes in engineering delivery, for CTOs and VPs of Engineering at startups and scaleups. Every number cited.en AI coding costs land in review, not on the invoicehttps://kaas.team/newsletter/p/ai-coding-costshttps://kaas.team/newsletter/p/ai-coding-costsWed, 07 Oct 2026 00:00:00 GMTAI coding costs land in review, not on the invoiceAI coding costs per engineer are a small share of pay. Review time decides whether delivery rises, and how to measure it at 20 to 150 engineers.Your AI coding bill is paid in reviewOne agent-written change, from the invoice to the number worth dividing by.SpendA small share of the engineer's pay.GenerateMore and larger pull requests, the same day.WaitThe queue fills on your senior engineers.Ship or slipMore merges unreviewed; adoption tracks with lower stability.MeasureCost per merged change, review included.What did your last merged change cost, counting the review?Your AI coding costs keep climbing, and talk of usage pricing makes next year's number harder to guess. Engineering leaders keep asking two things: will this end up costing as much as an engineer, and what are we getting for it?Faros AI's 2025 telemetry study found that teams with high AI adoption merged 98% more pull requests, while PR review time rose 91%.The AI coding bill is a small share of an engineer's pay; what decides the return is review, and without strong tests and fast feedback, more code turns into instability. To show where the money goes, we follow one change at Acme Inc: an agent writes "retry failed card payments from a queue" in an afternoon.Per engineer, the AI coding bill is small, though the invoice is where most of the worry sits. Anthropic's figures for enterprise use of its coding agent and DX's all-in estimate of seat plus token spend both come to a modest monthly sum per engineer.Levels.fyi's 2025 report puts a US software engineer's total yearly pay at about 226 thousand dollars. Against that, a year at the top of the vendor's average range is a little over one percent, and a year at the top of DX's range about three percent.A year of AI coding spend per engineer, out of each dollar a US engineer is paidAnthropic average, upper endDX all-in estimate, upper endThe engineerThat is our arithmetic on total pay rather than loaded cost, so the true share is lower still. The vendor figure is an aggregate with no sample size, so treat both shares as rough; even rough, the bill is nowhere near the cost of the person.Caps may come, and a few heavy users can surprise you. At Acme, though, the token line for the payment-retry change is a rounding error on the month's invoice. On our arithmetic over a full working year, a whole active day of agent use at the vendor's average costs less than one hour of the engineer's pay at Levels.fyi's figure.The bill buys more code, but the evidence that it moves company delivery is thinner than the dashboards suggest. In a telemetry study of over ten thousand developers, Faros AI found that gains in team behavior do not scale when aggregated to the company. Across more than 400 organizations, DX found a median PR throughput gain of 7.76%: a real gain, and a modest one.LinearB's 2026 benchmarks point to wasted output: AI-generated pull requests are accepted far less often than manual ones. Code that never merges costs review time and ships nothing.of AI-generated pull requests are acceptedLinearB benchmarksNot every study agrees. DORA's 2025 survey finds a positive link between AI adoption and delivery throughput; the section on tests below covers what DORA says the return depends on.Back at Acme, the agent opens the payment-retry PR the same afternoon, and the dashboard counts a pull request, though nothing has shipped yet.The extra output stalls in review. In Faros AI's data, PRs grew 154% larger on average, and review time rose with them.High-adoption teams against low, Faros AIPull requests mergedPR review timeIn LinearB's separate data, AI pull requests wait 4.6x longer before anyone picks them up, though they are reviewed 2x faster once someone does. Addy Osmani, who spent over 14 years leading developer experience at Google and became a director at Google Cloud AI, named the limit in a January 2026 essay.PR volume rises at companies your size too: Synthesia's 118 engineers went all in on AI coding tools in November 2025. Its CTO, Peter Hill, told IEEE Spectrum that pull requests had risen 120 percent year over year. He gave the volume entering review but not the hours it took.Faros AI's head of product marketing, Naomi Lurie, argues that senior engineers absorb the review load because they catch what AI gets subtly wrong. Cui and colleagues recorded seniority in the Microsoft experiment. They found the AI assistant significantly raised task completion for recent hires and juniors, but not for senior, long-tenured developers. If those PRs go to seniors for review, the juniors' extra output becomes the seniors' extra review; that step is our reading, not something either study measured.None of this is on the invoice, and no study we found prices it. Price it yourself: one hour of your staff engineer against the agent's whole token line. At Acme, the payment-retry PR is far larger than the change needed. It waits days, then lands on the one staff engineer who knows the payment flow, the same person whose juniors fill her queue.Under high AI adoption, more pull requests merge with nobody looking, and DORA links adoption to less stable delivery. Faros AI's 2026 report found 31% more PRs merged with no review at all under high AI adoption. Faros does not say whether that is a count of merges or a share of them. On our reading, each unreviewed merge can move cost from today's reviewer to whoever handles the next incident.more pull requests merged with no review at allFaros AI telemetry, high vs low AI adoptionDORA's 2025 survey finds AI adoption still has a negative relationship with delivery stability. DORA reports the link, not its cause.GitClear's analysis of changed lines shows a longer trend: refactoring shrank over the years AI tools spread. The timing matches adoption, but GitClear does not show that AI caused it.of changed lines were refactoring in 2021GitClear, changed lines across many reposThe evidence is mixed. Jellyfish, looking at bug tickets and reverts, found no significant relationship with AI adoption. The studies measure different things in different ways, and we do not know why their results differ.At Acme, under deadline, the second review on the payment-retry PR is waved through. A retry loop double-charges a handful of customers on a Friday. The rework and the incident are part of what the change cost.DORA's 2025 report says that without strong automated testing, mature version control and fast feedback loops, more change volume leads to instability. DORA draws that from its survey, not from a controlled trial.DORA adds that the greatest return comes from internal platforms, clear workflows and aligned teams rather than from the tools themselves.Jellyfish's Nicholas Arcolano runs research at a vendor that sells engineering measurement. In a talk on his firm's data, he compared teams by how their code is spread across repos.Treat the size of any gain with care. Pooled field experiments at three companies found a 26.08% rise in completed tasks, with a standard error of 10.3%, so the true gain could be small.Acme's payment-retry change, with review grown to fitSpecThe engineer writes what the change must and must not do.Agent writesThe agent drafts the change and its tests against the spec.Tests gateNothing reaches a person until the tests pass.Size checkOversized changes are split before review.Review on a targetA reviewer picks it up within the agreed pickup target.MergeThe change merges with its review hours logged.Survives a monthNot reverted or rewritten, so it counts as delivered.So measure the return where it is decided. One vendor defines the unit as total fully loaded AI cost over a period, divided by the PRs that merged and survived. Survived means not reverted or substantially rewritten inside a churn window. Put review hours inside that cost; our next post takes the metric apart.Acme puts a size limit and a test gate in front of review, and sets a pickup target for agent PRs. It starts dividing all AI cost, review hours included, by changes that merged and stayed merged. The payment-retry change now reaches its staff engineer small, tested and on time.In a team of 20 to 150 engineers, the scarce part is the judgment of the few people who can say a change is safe. Our reading of the evidence is that every gain from cheaper code is capped by how fast that judgment can be applied. At Acme, the staff engineer who knows the payment flow was the limit before any agent arrived; the agent made the queue in front of her longer.AI made writing cheap and left verifying slow to reach, so the cost moved to the step nobody bills for. DORA's word for it is amplifier: AI magnifies the strengths of high performers and the dysfunctions of struggling teams. The evidence suggests that a team without good tests and small changes gets busier and less stable, and no faster.So the budget question is about how you design review and testing more than about seats and tokens. The invoice is the easiest part to read and the least useful part to manage.We read vendor telemetry studies, DORA's survey, three field experiments and practitioner surveys, kept only figures we could quote word for word from the source, and name each vendor that sells measurement or review tools.The AI coding bill is still a small share of an engineer's payMore code arrives, but company delivery often does not moveThe extra output piles up in review, on your senior engineersMore code merges unreviewed, and adoption tracks with less stable deliveryWithout tests and fast feedback, more code turns into instabilityCode was never the scarce part of your teamWrite down the sharePut last quarter's AI spend for each engineer beside your loaded pay. It is your baseline, and it is probably small.Split the review clockPull review pickup time and time in review for the last month, AI-assisted PRs apart from the rest.Gate before a personSet a size limit and a pickup target for agent PRs, and make tests pass before anyone reviews.Count reviews by nameCount the reviews each senior engineer did last month. If one name carries the load, spread it.Start cost per merged changeAll AI cost plus review hours, divided by changes that merged and were not reverted within a month.What did your last merged change cost, counting the review?Right now, we're not sweating the costs because we're trying to evolve best practices for the tools, but that has resulted in some devs really blowing through budget, so we may start instituting caps on spending.It's hard to keep our CFO supportive about investing in these tools because the productivity benefits have proven difficult to conclusively prove.When output increases faster than verification capacity, review becomes the rate limiter.What’s really interesting is this highly distributed case. There’s essentially no correlation here between AI adoption and PR throughput. And actually the, the weak trend that does exist is actually slightly negative.