Stop Watching Your Coding Agent: Build a System You Can Trust
Inspired by Lauren Tan's (SpaceX) talk.
Recently, I watched Lauren Tan's talk about Agents (SpaceX, working a lot with Cursor) and found it difficult to understand. So I rewatched and rewatched and tried to write my own conclusions from it. This article is the result of this attempt and I hope it will help you to get her idea quickly.
https://www.youtube.com/watch?v=KwOX7vJyoOk
Lauren Tan said in the talk she had reached the point where coding agents were merging pull requests while she slept.
One morning, around twenty PRs had already landed on main. She reviewed them afterward. They were good.
Her reported output had reached roughly a thousand PRs in a month.
The obvious question is:
What model was she using?
But that misses the most useful part of her talk. Lauren didn't discover a magical prompt. She built an environment in which an agent could make a mistake, notice it, investigate it, fix it, test the fix and prove that the result worked.
Her most important word wasn't agent, it was:
verification. And that changes how we should build software with AI.
The problem: you are probably still doing half the work
A typical coding-agent session looks something like this:
You: Fix the login bug.
Agent: Done.
You: opens browser
You: It still doesn't work.
Agent: Ah. I found the problem.
You: No, that's not it.
Agent: You're right. I found the REAL problem.
You: sends screenshot
Agent: Ah...The agent writes code.
But you are the testing framework.
You launch the application,
reproduce the error,
copy console messages,
explain what happened,
and check the fix.
That works with one agent becomes absurd with twenty.
Lauren's key insight was simple:
If the agent cannot verify its own work, the human remains the bottleneck!
So the first upgrade is not a smarter model, it is a better loop.
1. Give the agent one command for “done”
Suppose your project contains Python and React.
If you're using Python today, I'd make uv the default. It gives us reproducible project dependencies and lets the agent run tools through the project's environment without manually activating a virtualenv.
Create:
scripts/verify.shFor a modern uv project, it might contain:
#!/usr/bin/env bash
set -euo pipefail
echo "== Python lint =="
uv run ruff check backend
echo "== Python types =="
uv run mypy backend
echo "== Python tests =="
uv run pytest -q
echo "== Frontend lint =="
npm --prefix frontend run lint
echo "== Frontend tests =="
npm --prefix frontend test -- --run
echo "Verification passed."If you're still using a traditional pip/virtualenv setup, the equivalent is:
#!/usr/bin/env bash
set -euo pipefail
echo "== Python lint =="
python -m ruff check backend
echo "== Python types =="
python -m mypy backend
echo "== Python tests =="
python -m pytest -q
echo "== Frontend lint =="
npm --prefix frontend run lint
echo "== Frontend tests =="
npm --prefix frontend test -- --run
echo "Verification passed."Then make it executable via:
chmod +x scripts/verify.shAnd add to your Makefile:
verify:
./scripts/verify.shNow this sentence:
Make sure everything still works.
has become:
make verifyThat is a surprisingly important change.
Your agent can now run this loop without you:
inspect
↓
change code
↓
verify
↓
failure
↓
inspect
↓
change code
↓
verifyThe agent doesn't need to believe its solution works. Instead, it can simply check.
Lauren describes taking this much further: letting agents run applications, collect performance traces, inspect heap snapshots, control simulators and reproduce actual user behavior.
The general rule is:
Give the agent access to the same evidence you would use yourself.
For
- a web app: browser automation.
- an API: real test requests.
- a CLI: run the CLI.
- iOS: simulator.
- performance bugs: traces and profiles.
Reasoning proposes, and reality gets the final vote.
2. For bugs, demand proof before the fix
Here is one of the most useful habits you can adopt immediately.
Suppose this function is broken:
def parse_timeout(value: str) -> float:
if value.endswith("s"):
return float(value[:-1])
if value.endswith("m"):
return float(value[:-1]) * 60
return float(value)Someone reports:
250ms is interpreted incorrectly.Don't let the agent immediately edit the function. First make it write:
def test_parse_timeout_milliseconds():
assert parse_timeout("250ms") == 0.25Now run only that test.
With uv:
uv run pytest tests/test_timeout.py -qWith a traditional environment:
python -m pytest tests/test_timeout.py -qWe want to see it fail!
Then fix the implementation:
def parse_timeout(value: str) -> float:
if value.endswith("ms"):
return float(value[:-2]) / 1000
if value.endswith("s"):
return float(value[:-1])
if value.endswith("m"):
return float(value[:-1]) * 60
return float(value)Then run the same command again: Green.
Why insist on this? - Because otherwise an agent can produce a perfectly reasonable fix for a problem it never actually reproduced. - The stronger sequence is:
reported bug
↓
observed failure
↓
code change
↓
observed successThat one habit eliminates a surprising amount of AI-assisted wandering.
3. Give the agent an onboarding manual
We routinely point an agent at a repository containing 200,000 lines of code and then seem surprised when it makes strange architectural decisions.
A new human engineer would receive onboarding. - So should your agent.
Create herefore an AGENTS.md:
# Project
FastAPI backend + React frontend.
Python dependencies are managed with uv.
## Important directories
backend/app/api/ HTTP endpoints
backend/app/services/ business logic
frontend/src/features/ feature code
tests/ backend tests
## Commands
Fast Python tests:
uv run pytest -q tests/unit
Full verification:
make verify
Development:
make dev
## Working rules
Before editing:
1. Reproduce the problem.
2. Inspect the implementation involved.
3. Find similar existing code before creating a new pattern.
4. Identify or add a test.
Before completion:
1. Run relevant tests.
2. Run `make verify`.
3. Inspect `git diff`.
4. Report exactly what was verified.For an older pip-based project, simply document:
Fast Python tests:
python -m pytest -q tests/unitKeep AGENTS.md short. Do not place the history of Western civilization in it.
The useful question is:
What does the agent need to know almost every time it touches this repository?
Everything else can be loaded when necessary!
4. Turn recurring lessons into Skills
Lauren noticed something interesting while supervising agents closely.
They would make the same kinds of mistakes repeatedly.
One particularly dangerous behavior was confidently diagnosing bugs without actually inspecting the relevant code.
So instead of correcting the behavior forever, she encoded better procedures.
We can do the same.
Create:
skills/debug-with-evidence/SKILL.md# Debug with evidence
Before modifying production code:
1. Capture the exact symptom.
2. Reproduce it.
3. Find the narrowest failing case.
4. Inspect the code actually executed.
5. Form hypotheses only after gathering evidence.
6. Prefer experiments that distinguish competing explanations.
7. Add a regression test when practical.
8. Make the smallest justified fix.
9. Rerun the reproduction.
10. Run full verification.
For Python projects managed by uv, run Python tools with `uv run`.
Report:
- observed failure
- root cause
- evidence
- files changed
- verification performedNow you have converted experience into reusable process.
This is much more powerful than repeatedly prompting:
Please investigate more carefully.
A Skill can also contain scripts. For example:
#!/usr/bin/env bash
set -euo pipefail
echo "=== STATUS ==="
git status --short
echo
echo "=== RECENT COMMITS ==="
git log --oneline -10
echo
echo "=== DIFF ==="
git diff --stat
echo
echo "=== TESTS ==="
uv run pytest -q --tb=shortAnd, for traditional Python environments:
python -m pytest -q --tb=shortLet deterministic software do deterministic work.
Use the language model for the part that actually requires judgment!
5. Convert review comments into hard rules
Here is perhaps the highest-value trick in this article.
Suppose you repeatedly tell agents:
Frontend code must never import the database package.
After writing that comment three times, stop reviewing it manually.
Write a check:
if rg 'app\.database' frontend/src
then
echo "Frontend may not import app.database"
exit 1
fiPut it into CI.
Now the architecture itself says no.
Lauren pushes this idea quite far: her agent-oriented architecture uses CI, static analysis and dependency restrictions to mechanically reject patterns she does not want.
This leads to an excellent rule:
Every repeated code-review comment is a candidate for automation.
Examples:
"Don't import X here."
→ dependency check
"Every endpoint needs authorization."
→ middleware + test
"Don't forget to regenerate the schema."
→ CI check
"Every bug fix needs a regression test."
→ workflow rule
"Don't modify generated files."
→ generated-file checkPython gives us particularly nice tools for this.
For example:
uv run ruff check .
uv run mypy .
uv run pytestOr in an older environment:
python -m ruff check .
python -m mypy .
python -m pytestInstructions are soft. Compilers, tests and CI are much less negotiable - and hardcode your rules into your system.
6. Make the easiest solution the correct solution
Agents - like humans - like shortcuts.
Rather than fighting this, design for it!
Imagine this structure:
components/
services/
hooks/
types/
validation/
screens/One feature may be scattered across all six directories.
Now consider:
features/
├── billing/
│ ├── api.ts
│ ├── model.ts
│ ├── BillingPage.tsx
│ └── BillingPage.test.tsx
│
└── login/
├── api.ts
├── model.ts
├── LoginPage.tsx
└── LoginPage.test.tsxIf the agent works on billing, most of the relevant world is physically nearby.
Lauren described this principle beautifully:
make the shortest path the best path.
If an agent naturally takes the easiest implementation route, build the repository so that the easiest route is also architecturally sound.
Incidentally, humans tend to enjoy such codebases too.
7. Use a fresh agent as reviewer
Don't always let the agent that wrote the code decide whether the code is good.
Use:
Agent A
↓
implements
↓
Agent B
↓
reviews from fresh contextGive the reviewer a simple checklist:
Check:
1. Does the change actually satisfy the task?
2. Can you reproduce the original bug?
3. Are edge cases missing?
4. Were tests weakened?
5. Is there unnecessary complexity?
6. Are architectural boundaries violated?
7. Is existing functionality duplicated?
8. Do the tests verify behavior?
For Python changes, run the relevant checks yourself:
uv run ruff check .
uv run mypy .
uv run pytestFor traditional Python use these equivalent commands instead:
python -m ruff check .
python -m mypy .
python -m pytestAnd require the reviewer to distinguish:
confirmed defect
plausible concern
stylistic preferenceOtherwise AI review can become surprisingly enthusiastic about redesigning perfectly acceptable code.
Fresh context is useful precisely because the reviewer does not know what the implementer meant. It sees what was actually written.
8. Parallelize with worktrees, not chaos
Once one agent works reliably, parallel agents become interesting.
Use Git worktrees:
git worktree add ../app-auth -b agent/auth
git worktree add ../app-search -b agent/search
git worktree add ../app-billing -b agent/billingNow three agents can work independently:
app-auth/
app-search/
app-billing/And if this is a uv project, each agent can simply work from its own checkout and run:
uv sync
uv run pytestNo manual virtualenv activation needed, and no wondering whether the shell is currently using the environment from the repository next door.
For pip-based projects, the equivalent setup might be:
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m pytestThis is one reason uv works particularly nicely with agent worktrees: creating a fresh reproducible Python environment is cheap and explicit!
A good division:
Agent 1: investigate authentication bug
Agent 2: implement CSV export
Agent 3: profile search performanceA bad division:
Agent 1: refactor authentication
Agent 2: refactor authentication differently
Agent 3: rename files both others are editingThe trick is not maximizing the number of agents.
It is parallelizing independent work.
9. Treat every human correction as data
This is where the workflow starts compounding.
Whenever you need to intervene, ask why.
Agent lacked project knowledge?
→ improve AGENTS.md
Agent didn't know the procedure?
→ create a Skill
Bug escaped?
→ regression test
Same architectural mistake again?
→ CI/static rule
Task was ambiguous?
→ improve task template
Agent trusted its own solution too easily?
→ independent reviewerNow a failure does not only fix one PR, but it improves the environment for every future PR.
That is exactly how Lauren describes climbing her “trust curve”: initially she watched agents closely, discovered their failure modes, then converted those lessons into verification systems, Skills and constraints.
The loop becomes:
agent makes mistake
↓
human understands why
↓
lesson becomes process
↓
process becomes Skill/test/CI
↓
future agent avoids whole category of mistakeThat is where the real leverage appears.
The setup I would build first
You do not need an elaborate multi-agent platform.
For one real Python repository, I would begin with:
pyproject.toml
uv.lock
AGENTS.md
Makefile
scripts/verify.sh
skills/debug-with-evidence/SKILL.md
skills/review-change/SKILL.mdIf the project does not use uv yet, a clean start is:
uv initFor an existing project:
uv syncAdd development tools as project dependencies, for example:
uv add --dev pytest ruff mypyThen your normal commands become pleasantly boring:
uv run pytest
uv run ruff check .
uv run mypy .For readers on a conventional pip setup, the familiar equivalents remain:
pip install pytest ruff mypy
python -m pytest
python -m ruff check .
python -m mypy .Then use this workflow:
1. Investigate.
2. Reproduce.
3. Write failing test.
4. Implement smallest fix.
5. Run fast tests.
6. Run full verification.
7. Fresh agent reviews diff.
8. Human corrections become permanent rules.A good everyday instruction can then be remarkably short:
Own this task end to end.
Before editing:
- inspect the relevant implementation,
- reproduce the problem,
- examine similar existing code.
During implementation:
- make the smallest coherent change,
- add or update tests,
- use `uv run` for Python tools,
- verify while iterating.
Before completion:
- run full verification,
- inspect the final diff,
- independently check the original requirement.
Report what changed, what was verified,
and any remaining uncertainty.The prompt is no longer carrying the whole engineering organization on its back.
The repository is doing much of the teaching.
The bigger idea
Lauren compares the future engineer partly to an engineering manager and partly to a head chef.
The head chef doesn't personally prepare every plate.
He
- designs the kitchen
- decides where things belong
- establishes procedures.
- checks quality.
- organizes parallel work.
- makes mistakes visible.
- creates an environment where doing the right thing is easier than doing the wrong thing.
That is a useful way to think about coding agents.
The important system is no longer just:
prompt → codeIt is:
requirement
↓
agent
↓
code
↓
execution
↓
verification
↓
review
↓
feedback
↓
better Skills / tests / architecture
↺A slightly weaker model inside an excellent engineering loop can be more useful than a brilliant model working blind.
And that leads to the experiment I would actually try: Take one annoying bug in one real repository.
Before asking an agent to fix it:
- create
AGENTS.md, - create
make verifyin aMakefile, - create a debugging Skill,
- make it reproduce the bug,
- make it write the regression test,
- let it implement the fix,
- let a fresh agent review it.
Hardcode what comes you into your way!
For a modern Python project, the inner loop should be as simple as:
uv run pytest tests/test_bug.py -q
uv run ruff check .
uv run mypy .
make verifyIf you're still using pip (but you should use uv!):
python -m pytest tests/test_bug.py -q
python -m ruff check .
python -m mypy .
make verifyWhenever something goes wrong, don't only correct the current agent, but ask:
What can I change so that this entire category of mistake becomes harder to make next time?
A test? Skill? CI rule? Feature map? Better architecture?
That is where coding agents become much more interesting than autocomplete.
The goal isn't twenty agents generating twenty times more code, but:
building a system in which twenty agents can eventually do useful work without requiring twenty times more supervision.