How to Prepare Your Codebase for AI Coding Agents

How to Prepare Your Codebase for AI Coding Agents

How to reduce guesswork, improve code reviews, and get more from AI coding agents.

Erfan Hesami and Pipeline to Insights

Oct 25

We plug an agent into the repo, describe what we want, and let it generate the code. That’s usually where the caution stops.

Faros AI tracked two years of activity across 22,000 developers and found that under high AI adoption, time to get a first PR review increases by 156.6%. Code churn, how much code gets rewritten after the fact, goes up 861%¹. Stack Overflow’s latest developer survey supports this from a trust perspective. 84% of developers already use AI coding tools. Only 3% say they trust the output².

More code moves faster, more time is spent reviewing it, and confidence in any of it drops once it ships. That’s not the trade most of us signed up for when we started delegating work to an agent.

Fiona Fung, who leads engineering and product for Claude Code and Cowork at Anthropic, has a one-line explanation for why these numbers look the way they do. The bottleneck has moved³. Writing code, writing tests, and refactoring used to be the slow, protected part of the job, and most of our process existed to protect that scarce resource. None of that is scarce anymore. What’s scarce now is verification: is the output correct, who checks it and how do we maintain it? The effort moved downstream, and most teams haven’t updated their processes to match.

Edward Donner, who teaches AI engineering through his own YouTube courses, names the same shift from a different angle. These tools make it fast to generate code and slow to review it, an asymmetry he thinks is quietly turning a lot of us into professional code reviewers⁴.

The usual response to this gap is a single file. Add an AGENTS.md, write down some rules, and the agent will finally understand our project. That advice is everywhere right now but it’s incomplete. An instruction file can only describe a project that’s already legible. It can’t invent consistency a codebase doesn’t have, and recent research suggests it doesn’t even describe as reliably as the hype implies.

In this post, I cover:

• Why AI coding goes wrong, and what it costs a team

• The AI Layer: a Third Item in Source Control

• A Better Workflow for AI Coding

Why AI Coding Goes Wrong

Here’s the pattern most teams fall into first, described by Cole Medin, who runs the Dynamous engineering community and has walked dozens of organisations through this exact transition⁵. A product manager writes a PRD (Product Requirements Document). They split it into tickets by hand. A developer opens one of those tickets, copies the description, pastes it into a coding agent, and types something like “implement this.” No conventions, no connection to the ticketing system, no shared process between developers.

The agent will usually produce something, but the problem shows up afterwards. It won’t follow the team’s own patterns because nothing told it those patterns exist. It can’t comment back on the ticket or open the pull request itself, because nothing connected it to those tools. What looks like a fast first draft becomes a slow review because the reviewer has to catch everything the agent couldn’t know.

This is the mechanism behind the numbers in the introduction, not a separate problem. The code gets written quickly. Understanding whether it’s right doesn’t travel with it, so review has to do all the work. Faster generation and slower review are the same event, viewed from both ends of the same pipeline.

Prithveesh Goel, a Principal Engineer at Expedia who has led AI adoption across more than 500 engineers, explains the goal simply. A repository should be organised the way we’d onboard a new engineer with a perfect memory but no knowledge of the company or its ways of working⁶.

That’s a high standard. Many repositories, even those that have been running in production for years, don’t meet it. Over time, human engineers can learn the missing details by asking colleagues and observing how the team works.

AI agents don’t have that same experience. Every new session starts without the knowledge that isn’t written down. When an agent doesn’t know a project’s conventions, it guesses. The result might look correct at first glance but still go against the way the team expects things to be done.

Example: Goel describes asking an agent to add a feature to a GraphQL codebase. Instead of using the project’s established query pattern and service layer, it wrote a raw JSON fetch directly inside a UI component. Nothing was technically broken. Nothing in the repo told the agent that pattern existed, so it reached for the closest thing it already knew!

The AI Layer: a Third Item in Source Control

Medin’s framing for what’s changed is the clearest one we’ve come across:

Before generative AI, your source control had two things: your code and your tests. Now there’s a third: the AI layer.

Medin and Goel work at different companies, but they independently came up with almost the same six parts of an AI setup:

1. General rules: instructions in AGENTS.md or CLAUDE.md that apply across the project.

2. Specific rules: instructions for a particular folder, part of the code, or type of task.

3. Skills: reusable workflows that tell an agent how to do common tasks, such as planning a feature or checking its work.

4. Sub-agents: smaller AI agents that handle narrower tasks the main agent delegates to them.

5. MCP servers: connections that let agents work with tools such as Jira, Confluence, and data warehouses.

6. Hooks: actions that run automatically when something happens, such as a tool call or the end of a session.

Both engineers arrived at almost the same list without comparing notes. That makes their approach worth paying attention to.

Fung’s team follows a similar idea on a smaller scale. If something is important to how a team works, such as a specification, process, or coding rule, save it in the repository as a skill or instruction file. It shouldn’t live only in a wiki that the agent cannot access, or that might be out of date.

The AGENTS.md format is now used beyond any single company’s tools. The Linux Foundation's Agentic AI Foundation oversees it. More than 60,000 repositories use it, and tools such as Claude Code, Cursor, Codex, and Windsurf can read it⁷. The format is a simple Markdown file with no required sections, but how you organise the instructions is important.

For example, OpenAI’s repository has 88 separate AGENTS.md files, with each in a different package. An agent reads the instructions closest to the code it’s working on first. The main file covers rules for the whole project, while files inside specific folders cover rules for that part of the code.

This organisation is better than a single instruction file. One large file containing every rule can overwhelm the agent with information it doesn’t need for the current task. Keeping general rules at the top level and specific rules close to the relevant code helps the agent find the right instructions when it needs them.

Does the Instruction File Even Work?

A team at ETH Zurich tested this question using AGENTbench, a set of 138 real-world Python tasks from different repositories. They tested four AI coding agents in three situations: without an instruction file, with one written by AI, and with one written by a developer⁸.

The results were mixed.

• Developer-written files improved task success by about 4% on average.

• AI-generated files made results about 3% worse.

• Both types increased AI usage costs by more than 20%.

The researchers found that agents followed instructions but spent extra time reading and checking without improving results. In their tests, the files did not provide a useful overview of the codebase.

However, there’s another side to this. Daniel Vaughan points out that the study tested individual tasks, not the long-term benefits of using the same rules across a team’s work⁹.

For example, a rule might save an engineer five minutes every week by preventing the agent from repeatedly discovering the same conventions. Over a year, those small savings can add up, even if they don’t improve a single task’s success rate.

Both findings can be true. A general instruction file that repeats what an agent can already learn from the code might add little value. A short, specific file that captures important project rules and stays up to date could be much more useful. The study doesn’t prove that all instruction files are ineffective.

There’s another reason to be careful when judging what AI agents can do.

Many charts showing how long AI agents can work independently use a 50% success rate. That means the agent completes tasks of that length successfully only half the time. METR’s research shows that agents typically achieve an 80% success rate on tasks that are four to six times shorter¹⁰.

This is important because completing a task sometimes is very different from being reliable enough to let an agent work on its own.

The same lesson applies to instruction files. We need to look beyond whether they exist or whether an agent follows them. What matters is whether they improve results, reduce repeated work, and make the agent more reliable.

Preparing a Data Pipeline for AI Agents

Most content stops at generic advice. The parts that are specific to our work are where it gets more useful.

dbt Labs has published its own guidance on what an agent-ready dbt project requires, and it comes down to four specific structural pillars¹¹:

1. Testing and validation infrastructure: tests, contracts, and CI that catch a problem early, the same checks we’d want regardless of who or what wrote the model.

2. Centralised business logic: definitions that live in the Semantic Layer instead of scattered across dashboards, SQL comments, and old Slack threads an agent can’t read.

3. Complete metadata: lineage, ownership, and documentation on every model, so an agent can answer “what depends on this” without guessing.

4. Local compilability: a project that compiles cleanly without warehouse compute, so the validation feedback loop stays fast.

Example: without the Semantic Layer pillar, an agent asked to add a churned flag has no single source for what “churned” means in this business. It’ll infer a definition from whatever SQL it reads first, and that definition might contradict the one three other models already encode.

dbt Labs’ own dbt-mcp repo carries this into code, with working agent integrations (CrewAI, LangGraph, and others) wired up against real dbt projects, if you want to see it rather than just read about it.

A second dbt Labs post, from the team’s developer experience group, adds behavioural guidance that holds up regardless of tool: explore before changing (read the docs, run a dbt show query before touching a model), preview real data before writing a test (so the test checks a real value instead of a hallucinated one), and work the DAG one model at a time instead of generating ten and declaring victory¹².

Naming discipline matters here as much as it does in application code, maybe more, because a data model’s name is often the only signal of what it means. One practitioner’s example, outside the data-engineering space but directly applicable, is naming a feature once and using that exact name everywhere, services, files, functions, so a prompt like “add AWS support alongside GCP in this feature” resolves to the right files instead of a guess¹³. The same logic applies to a stg_orders model and everything downstream of it.

On how long an instruction file should be, the numbers from our sources cluster rather than agree exactly: Goel’s own repos run around 60 lines; Medin recommends staying under 300; a separate AGENTS.md guide suggests a 200-to-500-line range¹⁴. Practitioners converge on the same order of magnitude, not one exact number: short enough not to overload the context window with irrelevant history, and specific enough to carry real, current conventions instead of restating what a careful read of the code would already show.

A Better Workflow for AI Coding

Structure alone doesn’t tell an agent how to work a single ticket. For that, Medin’s research-plan-implement-validate loop is the most specific process across everything I have found so far:

Research:

Pull the ticket and any linked documentation. Let the agent explore the relevant part of the codebase before anyone commits to an approach. This is the same step Goel’s prime skill performs, and the same instinct behind dbt’s “explore before changing” guidance. Three sources, one habit.

A step worth calling out on its own: before planning, have the agent interview you. Ask it to list its open questions about the ticket, then answer them, ideally through a structured format rather than free text.

Medin’s stated rule for this stage is the most useful line from my research. The number one job when working with a coding agent is to reduce the number of assumptions it makes. Every question answered now is one less thing the agent has to guess later, reducing the risk of code that looks right but is wrong.

Plan:

Turn the now-better-informed conversation into one structured document: the user story, the problem statement, the exact files and context involved, a task list, and, critically, the validation strategy written into the plan itself, which tests to run, what to check manually. The plan is reviewable before a single line of implementation code exists, just as a design doc is reviewable before a sprint starts.

Edward Donner adds a sharper distinction for what the spec itself should be. It’s intent, constraints, and success criteria, not a detailed requirements document the implementation has to match line for line¹⁵. His own demo repo carries this out end to end, spec through merged PR, if you want to see the whole loop worked through once.

When two engineers disagree on the right approach, Fung’s team uses a rule worth borrowing. Building is cheap, arguing is expensive¹⁶. Rather than settling it on a whiteboard, generate two or three real implementations and compare the actual diffs. The agent can make a disagreement concrete faster than either engineer can argue it in the abstract.

Implement:

Hand the plan to a fresh agent session rather than continuing the planning conversation. The reasoning is about context, not ceremony. A long planning conversation has already spent a meaningful share of its context window on research and back-and-forth that the implementation step doesn’t need. A clean handoff document carries everything required without carrying the clutter.

Validate:

The agent runs its own tests and lints, iterating until they pass. A human still does manual testing and a code review before opening a pull request. Full autonomy, prime straight through to a merged PR with nobody checking the plan or the diff, is possible once a team has enough history to trust its own system. Every source I checked for this post recommended keeping both human checkpoints by default rather than removing them early.

How far to push that autonomy is a project-by-project call, not a universal setting. Donner’s own rule of thumb is simple. A personal project or prototype can run on spot-checks and a glance at the results; something like a payments platform or a system handling health data gets a human reviewing every line, or no agent in the loop at all¹⁷.

Example: a ticket to add CSV export to a scheduling app’s meetings page. Research pulls the ticket and scans how export already works elsewhere in the app. The interview step surfaces a real decision point, should the export include cancelled meetings, which the ticket itself never specified. The plan documents the answer, the files to touch, and a validation step that downloads the CSV itself and checks its contents, not just that the button renders. Implementation runs in a clean session against that plan. Validation catches a regression in an unrelated test before a human ever opens the diff.

Keeping Your Setup Up to Date

An instruction file can become outdated as the code changes. Goel, Medin, and Fung suggest three ways to keep it useful.

1. Update it regularly

Goel uses a CI job to review recently merged pull requests and suggest updates to the instruction files. These changes are submitted as pull requests and require human review before they are merged.

2. Learn from mistakes

Medin’s approach is to update the instructions when something goes wrong. Instead of just fixing a bug, ask why the agent made the mistake and what rule or skill could have prevented it. Then submit that improvement as a pull request, just like a code change.

3. Remove what is no longer needed

Fung’s team also focuses on removing outdated rules and processes, not just adding new ones. For example, a team might have several overlapping rules for handling critical bugs and incidents, leaving people unsure which one to follow. Regularly review your rules and remove those that no longer serve a purpose¹⁸.

These three approaches solve different problems. Regular reviews catch changes over time, learning from mistakes helps prevent repeated errors, and removing outdated rules keeps the system manageable.

But does making your repository ready for AI agents mean you need to restructure everything? Not necessarily!

Denys Linkov, who leads machine learning at Wisedocs, spent six months moving a legacy medical-claims pipeline spread across ten repositories into a single repository. He later shared his experience and reflected on whether the effort was worthwhile¹⁹.

He compares technical debt to financial debt. Both can grow over time. The key question is whether the benefits of adding new features and serving more customers outweigh the cost of maintaining a complex system.

Sometimes, a full restructure makes sense. In other cases, restructuring just the part of the codebase an agent needs to work with is enough.

Linkov also found that newer AI models could handle the same refactoring task much more effectively than older models. An older model took three hours and made ten mistakes, while a newer one solved it with one correction. The newest model came close to solving it in one attempt.

This shows that better results don’t come only from improving the repository. AI models themselves are also getting better.

Good repository structure helps agents follow your team’s conventions, but it’s only part of the solution. The model’s capabilities and the codebase’s complexity matter too. The goal is to make practical improvements that deliver value, rather than restructure everything just because AI agents are becoming more capable.

Conclusion

We touched on this idea once before, in a single paragraph about designing a codebase so an agent could generalise from one example instead of reading the whole repository. It was true then, and it’s the same idea here, just with the rest of the argument attached. What makes a repository easier for an agent also tends to make it easier for a new human engineer, and none of Goel, Medin, or Fung builds anything more elaborate than that insight, repeated consistently and checked into source control.

Goel and Medin, independently, call what they’ve built a template rather than a framework, and push back on the notion of all-in systems that ask a team to change its entire make. That’s worth sitting with. None of the six components requires a rewrite. They require writing down what a careful engineer already knows about the project in a place an agent can read, and treating that document with the same discipline as the code it describes.

The smallest real place to start is to open whatever instruction file your repo already has and read it against what’s really true today. Wherever it’s wrong, fix that first, not the whole file; just make it accurate.

We value your feedback

If you have any feedback, suggestions, or additional topics you’d like us to cover, please share them. We’d love to hear from you!

Pipeline To Insights is where I share lessons, tutorials, and experiences in data engineering and AI to help you learn and grow.🙂

Your support helps me continue creating content like this, thank you 💐🙏

© 2026 Erfan Hesami
548 Market Street PMB 72296, San Francisco, CA 94104