Stop Watching Your Coding Agent: Build a System You Can Trust

Stop Watching Your Coding Agent: Build a System You Can Trust
Photo by ThisisEngineering / Unsplash

Inspired by Lauren Tan's (SpaceX) talk.

Recently, I watched Lauren Tan's talk about Agents (SpaceX, working a lot with Cursor) and found it difficult to understand. So I rewatched and rewatched and tried to write my own conclusions from it. This article is the result of this attempt and I hope it will help you to get her idea quickly.

https://www.youtube.com/watch?v=KwOX7vJyoOk

Lauren Tan said in the talk she had reached the point where coding agents were merging pull requests while she slept.

One morning, around twenty PRs had already landed on main. She reviewed them afterward. They were good.

Her reported output had reached roughly a thousand PRs in a month.

The obvious question is:

What model was she using?

But that misses the most useful part of her talk. Lauren didn't discover a magical prompt. She built an environment in which an agent could make a mistake, notice it, investigate it, fix it, test the fix and prove that the result worked.

Her most important word wasn't agent, it was:

verification. And that changes how we should build software with AI.


The problem: you are probably still doing half the work

A typical coding-agent session looks something like this:

You: Fix the login bug.

Agent: Done.

You: opens browser

You: It still doesn't work.

Agent: Ah. I found the problem.

You: No, that's not it.

Agent: You're right. I found the REAL problem.

You: sends screenshot

Agent: Ah...

The agent writes code.

But you are the testing framework.

You launch the application,
reproduce the error,
copy console messages,
explain what happened,
and check the fix.

That works with one agent becomes absurd with twenty.

Lauren's key insight was simple:

If the agent cannot verify its own work, the human remains the bottleneck!

So the first upgrade is not a smarter model, it is a better loop.


1. Give the agent one command for “done”

Suppose your project contains Python and React.

If you're using Python today, I'd make uv the default. It gives us reproducible project dependencies and lets the agent run tools through the project's environment without manually activating a virtualenv.

Create:

scripts/verify.sh

For a modern uv project, it might contain:

#!/usr/bin/env bash
set -euo pipefail

echo "== Python lint =="
uv run ruff check backend

echo "== Python types =="
uv run mypy backend

echo "== Python tests =="
uv run pytest -q

echo "== Frontend lint =="
npm --prefix frontend run lint

echo "== Frontend tests =="
npm --prefix frontend test -- --run

echo "Verification passed."

If you're still using a traditional pip/virtualenv setup, the equivalent is:

#!/usr/bin/env bash
set -euo pipefail

echo "== Python lint =="
python -m ruff check backend

echo "== Python types =="
python -m mypy backend

echo "== Python tests =="
python -m pytest -q

echo "== Frontend lint =="
npm --prefix frontend run lint

echo "== Frontend tests =="
npm --prefix frontend test -- --run

echo "Verification passed."

Then make it executable via:

chmod +x scripts/verify.sh

And add to your Makefile:

verify:
	./scripts/verify.sh

Now this sentence:

Make sure everything still works.

has become:

make verify

That is a surprisingly important change.

Your agent can now run this loop without you:

inspect
  ↓
change code
  ↓
verify
  ↓


failure
  ↓
inspect
  ↓
change code
  ↓
verify

The agent doesn't need to believe its solution works. Instead, it can simply check.

Lauren describes taking this much further: letting agents run applications, collect performance traces, inspect heap snapshots, control simulators and reproduce actual user behavior.

The general rule is:

Give the agent access to the same evidence you would use yourself.

For

  • a web app: browser automation.
  • an API: real test requests.
  • a CLI: run the CLI.
  • iOS: simulator.
  • performance bugs: traces and profiles.

Reasoning proposes, and reality gets the final vote.


2. For bugs, demand proof before the fix

Here is one of the most useful habits you can adopt immediately.

Suppose this function is broken:

def parse_timeout(value: str) -> float:
    if value.endswith("s"):
        return float(value[:-1])

    if value.endswith("m"):
        return float(value[:-1]) * 60

    return float(value)

Someone reports:

250ms is interpreted incorrectly.

Don't let the agent immediately edit the function. First make it write:

def test_parse_timeout_milliseconds():
    assert parse_timeout("250ms") == 0.25

Now run only that test.

With uv:

uv run pytest tests/test_timeout.py -q

With a traditional environment:

python -m pytest tests/test_timeout.py -q

We want to see it fail!

Then fix the implementation:

def parse_timeout(value: str) -> float:
    if value.endswith("ms"):
        return float(value[:-2]) / 1000

    if value.endswith("s"):
        return float(value[:-1])

    if value.endswith("m"):
        return float(value[:-1]) * 60

    return float(value)

Then run the same command again: Green.

Why insist on this? - Because otherwise an agent can produce a perfectly reasonable fix for a problem it never actually reproduced. - The stronger sequence is:

reported bug
    ↓
observed failure
    ↓
code change
    ↓
observed success

That one habit eliminates a surprising amount of AI-assisted wandering.


3. Give the agent an onboarding manual

We routinely point an agent at a repository containing 200,000 lines of code and then seem surprised when it makes strange architectural decisions.

A new human engineer would receive onboarding. - So should your agent.

Create herefore an AGENTS.md:

# Project

FastAPI backend + React frontend.

Python dependencies are managed with uv.

## Important directories

backend/app/api/       HTTP endpoints
backend/app/services/  business logic
frontend/src/features/ feature code
tests/                 backend tests

## Commands

Fast Python tests:

    uv run pytest -q tests/unit

Full verification:

    make verify

Development:

    make dev

## Working rules

Before editing:

1. Reproduce the problem.
2. Inspect the implementation involved.
3. Find similar existing code before creating a new pattern.
4. Identify or add a test.

Before completion:

1. Run relevant tests.
2. Run `make verify`.
3. Inspect `git diff`.
4. Report exactly what was verified.

For an older pip-based project, simply document:

Fast Python tests:

    python -m pytest -q tests/unit

Keep AGENTS.md short. Do not place the history of Western civilization in it.

The useful question is:

What does the agent need to know almost every time it touches this repository?

Everything else can be loaded when necessary!


4. Turn recurring lessons into Skills

Lauren noticed something interesting while supervising agents closely.

They would make the same kinds of mistakes repeatedly.

One particularly dangerous behavior was confidently diagnosing bugs without actually inspecting the relevant code.

So instead of correcting the behavior forever, she encoded better procedures.

We can do the same.

Create:

skills/debug-with-evidence/SKILL.md
# Debug with evidence

Before modifying production code:

1. Capture the exact symptom.
2. Reproduce it.
3. Find the narrowest failing case.
4. Inspect the code actually executed.
5. Form hypotheses only after gathering evidence.
6. Prefer experiments that distinguish competing explanations.
7. Add a regression test when practical.
8. Make the smallest justified fix.
9. Rerun the reproduction.
10. Run full verification.

For Python projects managed by uv, run Python tools with `uv run`.

Report:

- observed failure
- root cause
- evidence
- files changed
- verification performed

Now you have converted experience into reusable process.

This is much more powerful than repeatedly prompting:

Please investigate more carefully.

A Skill can also contain scripts. For example:

#!/usr/bin/env bash
set -euo pipefail

echo "=== STATUS ==="
git status --short

echo
echo "=== RECENT COMMITS ==="
git log --oneline -10

echo
echo "=== DIFF ==="
git diff --stat

echo
echo "=== TESTS ==="
uv run pytest -q --tb=short

And, for traditional Python environments:

python -m pytest -q --tb=short
Let deterministic software do deterministic work.

Use the language model for the part that actually requires judgment!


5. Convert review comments into hard rules

Here is perhaps the highest-value trick in this article.

Suppose you repeatedly tell agents:

Frontend code must never import the database package.

After writing that comment three times, stop reviewing it manually.

Write a check:

if rg 'app\.database' frontend/src
then
    echo "Frontend may not import app.database"
    exit 1
fi

Put it into CI.

Now the architecture itself says no.

Lauren pushes this idea quite far: her agent-oriented architecture uses CI, static analysis and dependency restrictions to mechanically reject patterns she does not want.

This leads to an excellent rule:

Every repeated code-review comment is a candidate for automation.

Examples:

"Don't import X here."
→ dependency check

"Every endpoint needs authorization."
→ middleware + test

"Don't forget to regenerate the schema."
→ CI check

"Every bug fix needs a regression test."
→ workflow rule

"Don't modify generated files."
→ generated-file check

Python gives us particularly nice tools for this.

For example:

uv run ruff check .
uv run mypy .
uv run pytest

Or in an older environment:

python -m ruff check .
python -m mypy .
python -m pytest

Instructions are soft. Compilers, tests and CI are much less negotiable - and hardcode your rules into your system.


6. Make the easiest solution the correct solution

Agents - like humans - like shortcuts.

Rather than fighting this, design for it!

Imagine this structure:

components/
services/
hooks/
types/
validation/
screens/

One feature may be scattered across all six directories.

Now consider:

features/
├── billing/
│   ├── api.ts
│   ├── model.ts
│   ├── BillingPage.tsx
│   └── BillingPage.test.tsx
│
└── login/
    ├── api.ts
    ├── model.ts
    ├── LoginPage.tsx
    └── LoginPage.test.tsx

If the agent works on billing, most of the relevant world is physically nearby.

Lauren described this principle beautifully:

make the shortest path the best path.

If an agent naturally takes the easiest implementation route, build the repository so that the easiest route is also architecturally sound.

Incidentally, humans tend to enjoy such codebases too.


7. Use a fresh agent as reviewer

Don't always let the agent that wrote the code decide whether the code is good.

Use:

Agent A
    ↓
implements
    ↓
Agent B
    ↓
reviews from fresh context

Give the reviewer a simple checklist:

Check:

1. Does the change actually satisfy the task?
2. Can you reproduce the original bug?
3. Are edge cases missing?
4. Were tests weakened?
5. Is there unnecessary complexity?
6. Are architectural boundaries violated?
7. Is existing functionality duplicated?
8. Do the tests verify behavior?

For Python changes, run the relevant checks yourself:

    uv run ruff check .
    uv run mypy .
    uv run pytest

For traditional Python use these equivalent commands instead:

python -m ruff check .
python -m mypy .
python -m pytest

And require the reviewer to distinguish:

confirmed defect
plausible concern
stylistic preference

Otherwise AI review can become surprisingly enthusiastic about redesigning perfectly acceptable code.

Fresh context is useful precisely because the reviewer does not know what the implementer meant. It sees what was actually written.


8. Parallelize with worktrees, not chaos

Once one agent works reliably, parallel agents become interesting.

Use Git worktrees:

git worktree add ../app-auth -b agent/auth
git worktree add ../app-search -b agent/search
git worktree add ../app-billing -b agent/billing

Now three agents can work independently:

app-auth/
app-search/
app-billing/

And if this is a uv project, each agent can simply work from its own checkout and run:

uv sync
uv run pytest

No manual virtualenv activation needed, and no wondering whether the shell is currently using the environment from the repository next door.

For pip-based projects, the equivalent setup might be:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m pytest

This is one reason uv works particularly nicely with agent worktrees: creating a fresh reproducible Python environment is cheap and explicit!

A good division:

Agent 1: investigate authentication bug
Agent 2: implement CSV export
Agent 3: profile search performance

A bad division:

Agent 1: refactor authentication
Agent 2: refactor authentication differently
Agent 3: rename files both others are editing

The trick is not maximizing the number of agents.

It is parallelizing independent work.


9. Treat every human correction as data

This is where the workflow starts compounding.

Whenever you need to intervene, ask why.

Agent lacked project knowledge?
→ improve AGENTS.md

Agent didn't know the procedure?
→ create a Skill

Bug escaped?
→ regression test

Same architectural mistake again?
→ CI/static rule

Task was ambiguous?
→ improve task template

Agent trusted its own solution too easily?
→ independent reviewer

Now a failure does not only fix one PR, but it improves the environment for every future PR.

That is exactly how Lauren describes climbing her “trust curve”: initially she watched agents closely, discovered their failure modes, then converted those lessons into verification systems, Skills and constraints.

The loop becomes:

agent makes mistake
       ↓
human understands why
       ↓
lesson becomes process
       ↓
process becomes Skill/test/CI
       ↓
future agent avoids whole category of mistake

That is where the real leverage appears.


The setup I would build first

You do not need an elaborate multi-agent platform.

For one real Python repository, I would begin with:

pyproject.toml
uv.lock
AGENTS.md
Makefile
scripts/verify.sh
skills/debug-with-evidence/SKILL.md
skills/review-change/SKILL.md

If the project does not use uv yet, a clean start is:

uv init

For an existing project:

uv sync

Add development tools as project dependencies, for example:

uv add --dev pytest ruff mypy

Then your normal commands become pleasantly boring:

uv run pytest
uv run ruff check .
uv run mypy .

For readers on a conventional pip setup, the familiar equivalents remain:

pip install pytest ruff mypy

python -m pytest
python -m ruff check .
python -m mypy .

Then use this workflow:

1. Investigate.
2. Reproduce.
3. Write failing test.
4. Implement smallest fix.
5. Run fast tests.
6. Run full verification.
7. Fresh agent reviews diff.
8. Human corrections become permanent rules.

A good everyday instruction can then be remarkably short:

Own this task end to end.

Before editing:
- inspect the relevant implementation,
- reproduce the problem,
- examine similar existing code.

During implementation:
- make the smallest coherent change,
- add or update tests,
- use `uv run` for Python tools,
- verify while iterating.

Before completion:
- run full verification,
- inspect the final diff,
- independently check the original requirement.

Report what changed, what was verified,
and any remaining uncertainty.

The prompt is no longer carrying the whole engineering organization on its back.

The repository is doing much of the teaching.


The bigger idea

Lauren compares the future engineer partly to an engineering manager and partly to a head chef.

The head chef doesn't personally prepare every plate.

He

  • designs the kitchen
  • decides where things belong
  • establishes procedures.
  • checks quality.
  • organizes parallel work.
  • makes mistakes visible.
  • creates an environment where doing the right thing is easier than doing the wrong thing.

That is a useful way to think about coding agents.

The important system is no longer just:

prompt → code

It is:

requirement
    ↓
agent
    ↓
code
    ↓
execution
    ↓
verification
    ↓
review
    ↓
feedback
    ↓
better Skills / tests / architecture
    ↺

A slightly weaker model inside an excellent engineering loop can be more useful than a brilliant model working blind.

And that leads to the experiment I would actually try: Take one annoying bug in one real repository.

Before asking an agent to fix it:

  1. create AGENTS.md,
  2. create make verify in a Makefile,
  3. create a debugging Skill,
  4. make it reproduce the bug,
  5. make it write the regression test,
  6. let it implement the fix,
  7. let a fresh agent review it.
Hardcode what comes you into your way!

For a modern Python project, the inner loop should be as simple as:

uv run pytest tests/test_bug.py -q
uv run ruff check .
uv run mypy .
make verify

If you're still using pip (but you should use uv!):

python -m pytest tests/test_bug.py -q
python -m ruff check .
python -m mypy .
make verify

Whenever something goes wrong, don't only correct the current agent, but ask:

What can I change so that this entire category of mistake becomes harder to make next time?

A test? Skill? CI rule? Feature map? Better architecture?

That is where coding agents become much more interesting than autocomplete.

The goal isn't twenty agents generating twenty times more code, but:

building a system in which twenty agents can eventually do useful work without requiring twenty times more supervision.