Build a Definition of Done for Claude Code & Codex
Stop letting the agent grade its own homework. Drive it non-interactively with a loop that proves it finished with an exit code, not a feeling.
Greg Ceccarelli · Co-founder, SpecStory · July 14, 2026
I'm Greg. I ship with agents daily.
Co-founder of SpecStory, where we capture every AI coding session so you can search, share, and learn from them. Before that: CPO at Pluralsight, data at GitHub, Dropbox, and Google.
This lesson comes out of Hardcore Agentic Engineering, a course I teach with my co-founders. One conviction drives it: the hard part isn't getting agents to produce work. It's engineering the proof it's correct.
- Thousands of agent sessions captured, the same failure everywhere: "done" that isn't
- The fix is old, boring, and it works. We'll build it in ten lines
Everyone assumes the agent stops because the work is done.
It stops because it stopped calling tools.
// every item the model streams either // schedules a tool call, or doesn't (:2134) needs_follow_up |= output_result.needs_follow_up; if !needs_follow_up { break; // ← no tool call. the run ends. :373 } Ok(last_agent_message) // "done" is this string :460
The next forty minutes
One promise: by part four you have something you can run tonight.
A coding agent is a conversation with hands
Claude Code and Codex are terminal programs. You type a request in plain English; an AI edits your files and runs your commands to fulfill it.
Each turn, the model answers with one of two things: words, or a tool call: "run this command," "edit this file." The program executes the call and hands back the result.
> add a /healthz endpoint ⚙ tool call: Read src/app.ts ⚙ tool call: Edit src/app.ts ⚙ tool call: Bash npm test → 14 passing Done! Added /healthz and the tests pass. ← just words. no tool call.
Hold on to that last line: words, no tool call. It's about to matter a lot.
The model can't touch your computer. The harness can.
The model is a text function: it can ask to run a test; it can't run one. The harness is the program wrapped around it: it executes the tool calls, feeds results back… and decides when to loop and when to stop.
Claude Code · Codex CLI · Gemini CLI · Copilot CLI: different badges, same job. "Agent" = model + harness.
The loop you're already running
Claude Code, Codex, Gemini CLI, Copilot: different products, same while-loop. Reason, act, observe, repeat…
…and the run ends the moment the model replies without calling a tool. Nothing checks anything at that boundary by default.
"Done" is a feeling
The green checkmark is a string the model produced, not an event in the world.
The agent isn't lying. It's predicting the most plausible next sentence, and after a run that looked fine, "Done!" is extremely plausible.
The agent doesn't stop when it's done. It's "done" when it stops.
- The victory lap: "All tests passing! ✅" (it never ran them)
- The quiet shrink: scope trimmed mid-run so "done" arrives sooner
- The plausible summary: a beautiful description of a state the repo is not in
Definition of Done: the origin
Mid-2000s agile teams kept demoing work that was "90% done": coded but untested, unreviewed, undeployable. "Done-done" entered the vocabulary as a joke…
…and Scrum made it a discipline. The 2020 Scrum Guide elevates the Definition of Done to the formal commitment attached to every Increment.
Work that doesn't meet it isn't part of the Increment at all; it can't even be shown at Sprint Review. Meet it and, in their phrase, "an Increment is born."
- Code peer-reviewed
- Tests pass in CI
- Deployed to staging
- Docs updated
- No new lint errors
Why it worked, and why it transfers
- Written before the work: goalposts committed in advance
- A checklist, not a vibe: each item checkable by anyone
- Binary: there is no 90% done
- Shared: the worker doesn't privately define done
Scrum wrote the Definition of Done for humans who fool themselves.
Agents fool themselves faster. At 3 a.m. Unattended.
Increment ↔ the diff · developer says "done" ↔ model says "done" · DoD ↔ executable checks · Sprint Review gate ↔ the loop's exit condition
2026: the loop gets a target
Codex CLI ships /goal
A session-scoped completion condition, re-injected every turn with a built-in completion audit: "treat uncertainty as not achieved."
Claude Code ships /goal
Implemented as a Stop hook: each time Claude finishes a turn, the condition + conversation go to a small fast model that answers yes/no. No yes, no stopping.
Goal engineering
The practice around it: a checked-in goal (≤4,000 chars, the cap Codex enforces) + a rider with phases and named tests. The brief becomes an artifact with a SHA. gregceccarelli.com/goal-engineering
Real progress: the loop finally has a destination, and the harness won't let the model wander off. But look who decides you've arrived…
Who's grading the homework?
LLM-as-judge (/goal today) | Deterministic checks | |
|---|---|---|
| Who decides | A model reads the transcript, answers "done?" | A script exits 0 or non-zero |
| The verdict is | Opinion: probabilistic, persuadable | Evidence: binary, reproducible |
| Cost per turn | One more model call | Milliseconds |
| Fuzzy criteria ("prose is friendly") | Yes, its superpower | No, only what you encoded |
| Swayed by a confident summary? | Yes, it judges the same conversation | Never reads the transcript |
| Setup cost | Zero | You write the checks |
A judge answers "does this look done?" A check answers "is this done?"
Two files. That's the lesson.
#!/usr/bin/env bash set -e npm test # tests pass test -f dist/app.js # artifact exists grep -q '"/healthz"' \ src/app.ts # it's really there
#!/usr/bin/env bash until ./check.sh; do claude -p "Goal in TODO.md. Run ./check.sh, read the first failure, fix it, commit." \ --allowedTools "Bash,Read,Edit" \ --permission-mode acceptEdits done echo "DONE: check.sh exited 0"
until is the whole orchestration layer. The exit code is the Definition of Done. The model's opinion never touches the exit condition. Exit 0 = an Increment is born.
Using Codex instead of Claude? One change: the claude -p … line becomes codex exec --full-auto "…" with the same prompt.
$ ./loop.sh ── turn 1 ───────────────────────────── $ ./check.sh ✗ npm test 2 failing exit 1 → claude -p "…fix the first failure…" ⚙ read failures · edited src/date.ts · commit 9f3a21c ── turn 2 ───────────────────────────── $ ./check.sh ✓ npm test ✗ test -f dist/app.js missing exit 1 → claude -p … ⚙ ran npm run build · commit 4c81d02 ── turn 3 ───────────────────────────── $ ./check.sh ✓ npm test ✓ test -f dist/app.js ✓ grep -q '"/healthz"' src/app.ts exit 0
Done · exit 0 · stamped by the check, not the model
I didn't decide it was done. The model didn't decide. The exit code decided.
The loop, one step further
- ① It retries blind: the agent never hears why the gate said no. Pipe check.sh's output into the next prompt.
- ② It retries forever: no budget. Cap the attempts; NOT DONE is an honest result too.
- ③ It trusts the agent near the gate:
chmod 444 check.sh: fix the code, never the check.
#!/usr/bin/env bash chmod 444 check.sh # ③ lock the gate for attempt in 1 2 3 4 5; do # ② budget out=$(./check.sh 2>&1) && { echo "DONE on try $attempt"; exit 0; } # ① feed the failure back in: claude -p "Goal in TODO.md. check.sh failed: $out Fix the code, never the check. Commit." \ --allowedTools "Bash,Read,Edit" \ --permission-mode acceptEdits done echo "NOT DONE after 5 tries"; exit 1
Feedback → deadreckon's per-turn gate report · the budget → its spend caps & resumable turns · the lock → dr-gate. You just reinvented the next slide.
Same idea, industrial grade
| Your ten-liner | deadreckon.sh |
|---|---|
Agent could just edit check.sh | dr-gate: a watchdog outside the sandbox holds a per-run secret and re-runs the checks itself; the agent can't stamp its own pass |
| Runs loose in your repo | Isolated git worktree: your branch untouched until promotion |
| Dies with your laptop | Every turn snapshotted to disk; resume from the last completed turn |
| Leaves a transcript to take on faith | Evidence artifacts: run narrative, decisions, per-file provenance |
| You write checks by hand | deadreckon def-done "sign up, log in, save a drawing" → compiled to executable checks |
curl -fsSL https://deadreckon.sh/install.sh | sh · open source · MIT · supervises the CLI you already use (Claude Code, Codex, Gemini, Copilot…)
Say "done" in English. Get a gate.
$ deadreckon def-done "tests pass, dist/app.js exists, /healthz is served" compiling plain English → .deadreckon/acceptance.yaml checks: - kind: build_success # npm test, green - kind: file_exists # {working_dir}/dist/app.js - kind: content_match # "/healthz" in src/app.ts $ deadreckon start "Goal in TODO.md" dr-gate re-runs these checks itself and signs the pass. the agent can't reach either one. ✓ gate passed · run promoted
The same three check species you just wrote by hand in check.sh. Here the English is the source, and the gate lives outside the agent's reach.
When to skip the ceremony
- You're sitting there watching: you are the gate
- The output is learning, not an artifact: spikes, exploration
- "Done" is taste-shaped: naming, tone, design (a judge is right here)
- The change is smaller than the ceremony: a five-line tweak
- You're walking away: overnight, lunch, the weekend
- The task repeats: batch migrations, fleets of repos
- "Done" is fact-shaped: tests green, build passes, endpoint responds
- You intend to trust the result without reading the transcript
If you're watching, you are the definition of done. If you're walking away, write one a machine can check.
Five things we talked about
/goal gives the loop a target, but an LLM judges arrival: opinionuntil ./check.sh makes done an exit code: evidenceTonight, in three steps
- Hardcore Agentic Engineering · for builders who ship · Aug 3–21 on Maven · ten live sessions
- The full discipline: Brief → Steer → Verify. You build prove-it, a harness that survives
kill -9and gates "done" behind checks the agent can't reach - Enroll at get.specstory.com/agentic-engineering. It applies promo code HARDCORE for you
Two more lessons this month
Build an environment-aware AI Agent from Scratch
An agent harness isn't a black box. It's a loop you can build yourself in an afternoon. With John Berryman (GitHub Copilot; author, Prompt Engineering for LLMs).
Make Your Repo Agent-Ready: Rules, Docs, Reviews
Most agent failures aren't the model's fault. They're the repo's. With Dan Gerlanc (co-founder, .txt / Outlines).
- 25 Patterns in Agentic Engineering · free PDF
- The moves that ship software when the agent writes the code and the human owns the proof
- Drawn from 1,310 captured agent sessions & 4,670 commits
- specstory.com/books/25-patterns-in-agentic-engineering-book-2026.pdf
Watching the recording? Everything lives at maven.com/specstory: lessons, recordings, and the course.
Questions: put them in the chat
👍 the ones you want answered. Specific beats general. Bring your actual task.
- This deck + the demo
github.com/specstoryai/hardcore-agentic-engineering - Capture every agent session
github.com/specstoryai/getspecstory - The gate the agent can't reach
deadreckon.sh - Say hi
greg@specstory.com · @gregce10
Sources: deadreckon.sh · "Goal Engineering" & "The Green Checkmark is Not a Fact" (gregceccarelli.com) · Scrum Guide 2020 · Claude Code & Codex CLI docs