Building an AI-native engineering team by what you check

An AI-native engineering team is defined by what it checks, not what it uses. Four stages and the first change at each, for 20 to 150 engineers.

· 8 min read

Your AI stage is what you check

One agent-written change, from a private habit to a measured flow.

On your team, who checks the agent's work before a reviewer is asked?

KaaS Newsletterkaas.team/newsletter

A four-step track for one agent-written change: Private, where each engineer uses AI alone; Shared, with one context file and named boundaries; Checked, with a named owner and passing checks before review; Owned, with a review policy, where the agent carries the ticket and flow measures say if it held. Save the picture

Our whole team uses AI agents, so are we AI-native? That is the question every engineering leader gets, and the answers on offer are a seller's ladder or a tool timeline.

In a survey of one newsletter's readers, 95% use AI tools at least weekly1. When almost everyone is already there, usage cannot set a stage.

We argue a team's AI stage is better read from what it checks than from how much AI it uses, because each stage adds one habit of checking. Someone has to own and check what the agent produces. That is why building an AI-native engineering team comes down to what you check.

To see it, we follow one change at Acme Inc: the checkout squad's annual billing change. What differs at each stage is who wrote the context, who owns the result, and who reads it first.

Usage cannot separate teams when 55% of engineers already use agents

In that survey, 55% regularly use AI agents1, so the first transition, from autocomplete to agents, is already behind most of them. The sample is self-selected and does not single out teams of 20 to 150 engineers, so read it as a hint.

Eighteen months ago, AI usage was mostly for code generation and tab completion.

Gergely Orosz and Elin Nilsson, Survey authors, The Pragmatic Engineer1
55%

of surveyed engineers regularly use AI agents, up from mostly autocomplete eighteen months earlier

surveyed readers of one newsletter, a self-selected sample. Source: The Pragmatic Engineer, AI Tooling for Software Engineers in 20261

How fast it feels is not a signal either. In a small randomized trial on the early tools of its day, developers who had been measured slower still believed AI had sped them up by 20%2. The trial is small and its page says a newer update supersedes it, so we use it for that one point.

At Acme, every engineer uses an agent, so by usage the answer to the opening question is yes. That answer says nothing about who checks the billing change before it merges. If the squad cannot name that person today, the billing change is sitting at the first stage whatever the dashboard shows.

Thanks for reading KaaS Newsletter

Subscribe for free to get new posts by email.

We email you each new post. You can leave at any time. Privacy

Output rises first, and company results do not follow adoption

Output rises on high-adoption teams, and across companies the vendor found no link between adoption and company results. In one engineering analytics vendor's data, developers on teams with high AI adoption merge 98% more pull requests3.

Change on teams with high AI adoption3
  • Tasks completed+21%
  • Pull requests merged+98%

Merges rose 98%3 and review time rose 91%3. Our reading: the review load grew nearly as fast as the output, so reviewers carry almost as much more as the team ships.

Across thousands of developers the vendor observed no significant correlation between AI adoption and improvements at the company level3. This is one vendor's telemetry and it shows association, not cause.

we remove friction from some places like code authoring and testing, but then add friction in other places

Laura Tacho, CTO, DX4

At Acme, the agent writes the annual billing change quickly, and a human still has to review it. The change is bigger than the ticket that asked for it, so it lands in the senior engineers' queue and waits there. Speed at the keyboard has turned into waiting in review.

The four stages are four answers to who checks the agent's work

Each stage answers who checks the agent's work: the engineer, the squad's file, the owner, or a flow measure. The stages are ours, not published. We found no model that stages teams by what they do. The ones on offer are a seller's ladder and a tool timeline that never defines the term.

Checked rests on review being the limit. One engineering newsletter's author argues that building is now limited by how fast we can review the generated code5. We take that as the view behind Checked, not a measured result. Owned is our hypothesis, with no source behind it.

Acme's annual billing change, at each stage
  1. PrivateUses the agent alone, with nothing written down.Engineer
  2. SharedThe agent works from a shared context file with boundaries.Squad
  3. CheckedChecks pass and the owner explains the change before review.Owning engineer
  4. OwnedThe agent takes the whole ticket under a review policy, and flow measures decide if it held.CTO

You can read your own stage off your pull requests. At Private, nothing about AI is written down in the repo. At Shared, a new engineer's agent behaves like everyone else's.

At Checked, review time rather than generation sets the pace. At Owned, stability and wait time hold while the merge count rises.

you are not just a user of the tool; you are the ultimate guarantor.

Addy Osmani, The AI-Native Software Engineer, Elevate6

That is the individual version of the third stage. The publisher of the DORA research writes that AI doesn't fix a team; it amplifies what's already there7. Our read: a team with no habit of checking amplifies unchecked work.

The same publisher reports that AI adoption still has a negative relationship with delivery stability7. That is adoption alone. It does not show that ownership reverses it.

Step back and ask why the stages exist. Generation got cheap, so usage stopped being evidence of anything. Nobody needs a dashboard to prove a team uses the tool. Verification stayed expensive.

Each stage answers who pays for it: the engineer alone, the squad's shared file, the owner before review, or a flow measure the leader reads. We think verification is the cost that stays. That is our reading, not a measured result.

At each stage the first change is a habit, not a tool

Each step up adds one habit, and none of them is a purchase. Moving from Private to Shared starts with a short context file. One code host's guidance, drawn from many public repositories, says to cover six areas8.

Six

areas the guidance says to cover, from over 2,500 repositories

one code host's guidance on agent context files. Source: GitHub Blog, How to write a great agents.md: Lessons from over 2,500 repositories, 20258

Keep the file small. The same guidance says to start simple, test it, and add detail when the agent makes mistakes8. When the agent errs, a line goes in, and the check passes next time.

The coding agent's own guidance says the agent reports done when it had no way to check its own work9. We propose that moving from Shared to Checked means a runnable check for every task, with the exact test commands in the context file. The same guidance says a rule the agent keeps missing belongs in a hook, not another line in the file9.

For Checked, every agent change meets three gates before review: checks pass, then the owner explains it, then a human reviewer is asked.

We also propose that moving from Checked to Owned starts with a one-page review policy with explicit risk tiers. That is advice written for managers by one fractional CTO, not a report of what a company did10. The first change at each stage is our judgement; no source ranks them.

Before the billing change, Acme's squad writes a context file with the test command and a boundary around the payments folder. The agent now fails a check instead of reporting done. That is the whole move at this step: no new tool, one file the squad can read and one command the agent can run. Whether the billing change is then right is still for the owner of the change to say.

The top stage is hardest to read because agents own whole tickets

Stage four is where the evidence thins, and we say so. We found no source that gives an observable signal for agents owning whole tickets, and no leader at a company of this size describing the move. What we have is one engineer's remark and one consultant's question, so the judgement here is ours.

It's like having a distributed team of junior developers, each working to my guidance.

Unnamed engineer, Engineer, One software company11

This is an engineer at one company, who is not named on the post, and the remark is a single anecdote about directing agents. It does not show agents owning whole tickets. It does hint at the owner's job: direct the agents and read what comes back. In our design the owner reviews the result before merge.

It changes the leader's job too. For an owned ticket, the leader reads flow measures to learn whether it held. We would read review wait and revert rate next to the merge count. That choice is ours, and no source we read recommends those two measures.

If agents do the work juniors learned on, where do the next seniors come from?

Marian Kamenistak, Fractional CTO and engineering leadership mentor, marian.coach10

Stage four leaves that question open, and no source we read answers it. At Acme, the annual billing change is the last ticket the team lets an agent carry end to end, and the CTO watches the flow measures before letting it carry the next.

What to do on Monday

  1. Name your stage, this week. Open the last ten merged agent changes. Is there a named owner, a passing check before review and a written boundary? That is your stage, not the usage dashboard.
  2. Write a context file, this week. Cover commands, testing, structure, style, git workflow and boundaries, and add a line only when the agent makes a mistake.
  3. Check every task, this week. Put the exact test commands in the file, and move any rule that must not bend into an enforced check.
  4. Write a review policy, next week. One page with risk tiers for agent-written code, as the step from Checked toward Owned.
  5. Pair the merge count, next week. Put a flow measure such as review wait or stability beside it before you read a gain as real.

The companion, Build an AI-Ready Engineering Team, is free with your work email.

See the Agentic SDLC masterclass

On your team, who checks the agent's work before a reviewer is asked?

Param

Subscribe to KaaS Newsletter

Research on what AI changes in engineering delivery, for CTOs and VPs of Engineering at startups and scaleups. Every number cited.

We email you each new post. You can leave at any time. Privacy

Sources

  1. The Pragmatic Engineer, AI Tooling for Software Engineers in 2026, 2026
  2. METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, 2025
  3. Faros AI, The AI Productivity Paradox, 2025
  4. The Pragmatic Engineer, how tech companies measure the impact of AI, 2025
  5. Engineering Leadership, Code Review is the New Bottleneck For Engineering Teams, 2026
  6. Addy Osmani, Elevate: The AI-Native Software Engineer, 2025
  7. Google Cloud blog, Announcing the 2025 DORA Report, 2025
  8. GitHub Blog, How to write a great agents.md: Lessons from over 2,500 repositories, 2025
  9. Anthropic, Best practices for Claude Code, 2026
  10. marian.coach, AI Agents Don't Shrink Engineering Teams. They Expose Weak Managers., 2026
  11. incident.io blog, shipping faster with Claude Code and Git worktrees, 2025

KaaS Newsletter

Research on what AI changes in engineering delivery, for CTOs and VPs of Engineering at startups and scaleups. Every number cited.

We email you each new post. You can leave at any time. Privacy

See all posts

Bring us the work that waits.

A 30 minute call with the founder about where AI should start in your company.