The pitch for coding agents is that you describe the task once and walk away. The usual experience is different. Twenty minutes into the run, the agent stops to ask how you want errors handled, and the task joins the pile of work that only progresses while you watch. **Supervision is not a property of the agent, it is a property of the workflow around the task.** The same model that interrupts you every ten minutes can finish alone when the task arrives with the answers it would otherwise ask you for. This article is about the workflow changes that make that happen. ## The Questions You Keep Answering Every mid-task interruption is one of three questions. What does done look like? What is true right now? What would you do here? An interactive session hides how much of the task you are supplying, because you answer these questions in seconds and the work keeps moving. The task completes, and the completion feels like proof the agent could do it. It is proof only that the two of you could do it, together. That distinction matters because a task that finishes only while you answer questions was never delegated, it was paired, and pairing does not scale. The cost of live supervision is the cost I worked through in [Managing Many Concurrent LLM Agent Sessions](../managing-many-llm-agent-sessions/index.md): working memory holds a handful of sessions at best, every switch leaves attention residue, and a dozen parallel tasks leave you answering one question at a time. **Every interruption is a question your workflow failed to answer before the run started.** The fix is not a better model, it is moving the answers out of the session and into the workflow. ## Pay the Specification Cost Up Front The interactive workflow lets you defer writing down what you want, because you can supply it live forever. Delegation removes that option. Before you hand off a task, write four things: the outcome you expect, the constraints it must respect, the acceptance criteria that decide pass or fail, and what is explicitly out of scope. The acceptance criteria are the critical part of the handoff, and the test for them is simple. Could a competent colleague decide whether the finished work passes without contacting you? If the answer is no, the agent will end its run by asking you, one way or another. The text you hand over is also the only durable record of intent, a generalization of the argument in [The Prompt Is the Source Code](../the-prompt-is-the-source-code/index.md): for prompted code, the intent is in the prompt and the conversation around it, not in the code, which is only the compiled output. Commit the specification next to the work. **If acceptance of the finished work requires you, the task was never fully specified.** ## Replace Yourself With a Checker Mid-run, the agent cannot ask you what is true, so ground truth has to arrive as signals it can read on its own: the failing test, the type error, the lint violation, the broken link. This is the same move I described in [Verifying Code Without Reading It](../verifying-code-without-reading-it/index.md), applied one level up, to the task instead of the diff. An agent that can run the test suite can loop on its own mistakes without you. An agent that cannot run any check will report success based on its own judgment of its output, which is exactly the failure mode behind [The Acceptance Gap](../the-acceptance-gap/index.md). Testers have a name for the trap, the [oracle problem](https://en.wikipedia.org/wiki/Oracle_(software_testing)): a system that both produces an answer and judges it has no way to catch itself being wrong. That is exactly the situation of an unattended agent, and [Anthropic's guidance on building agents](https://www.anthropic.com/research/building-effective-agents) identifies ground truth from the environment as a core mechanism that lets an agent correct its own mistakes. **Every check you automate converts a question the agent would have asked you into a signal it can read itself.** The corollary is uncomfortable: tasks with no encodable oracle will keep requiring your involvement, so either build the checker before the handoff or accept, knowingly, that those tasks stay supervised. The cheapest checker to build first is a failing test: write the test that proves the bug, hand over the task, and let the agent iterate until it passes. ## Move Judgment Into Standing Documents The third question, what would you do here, has a different answer path, because judgment cannot be written into one task. It gets written into standing documents that every run reads before it starts: an AGENTS.md of conventions, skills that encode procedures, templates, decision records. In-session answers are lost when the session ends. An answer written into a document outlives it, and the loop compounds: an interruption happens once, you write the answer down where the next run will find it, and that question never gets asked again. This is the argument of [My AI Workflow](../my-ai-workflow/index.md), where the skills library, not the tools, is the part that keeps improving. I run a worked example of this practice on this very blog. The `agents/` section publishes AI-written articles with no per-article human review, held together by a rules file, mandatory disclosure tags, and a link-checking pipeline. I do not review each article before it ships, I edit the rules when an article goes wrong. **Supervision only declines if every answer you give outlives the session that prompted it.** ## Size the Task to the Horizon [METR's measurement of agent task horizons](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) put the frontier at roughly one hour of human task time at 50% reliability in early 2025, doubling about every seven months as of that measurement, and the 80% reliability horizon was far shorter. Humans still hold a decisive edge on long tasks. A professional can be handed a three-day assignment and be trusted to work through it. Hand a current model a three-day task, and the odds that it reaches the end without a mistake are poor no matter how well you specify it. So do not hand off a task at its natural size, hand it off at the model's size. Decompose until each unit fits inside the reliable horizon, make each unit independently verifiable, and make each unit reversible so a mistake costs minutes instead of days. Write the stop condition into the task itself: if blocked, or if the checks cannot be made to pass, stop and report rather than improvise. **Give the agent a task that fits its horizon, not a task that fits its description.** This is the small-scale version of the closed loop I described in [Zero Touch Engineering](../zero-touch-engineering/index.md): observe, decide, implement, verify, ship, learn, with the human outside the loop and re-entering it only through review. ## Move Your Attention to the Ends of the Run With specification up front and checkers in place, there is nothing useful for you to do in the middle of a run, so stop watching it. Watch the ends instead. When the run finishes, review the output, and when a run fails, triage the failure into one of three buckets. A specification gap: rewrite the spec so the next run does not miss the same thing. A missing checker: add the check so the failure becomes impossible to miss. A model limit: do that part yourself, or wait for the horizon to double past it. The first two buckets are permanent fixes, and they are why review-after beats steering-during: every reviewed failure upgrades the workflow for all future runs. Live steering feels productive, but your live attention is the scarcest resource in the system, the same queue dynamic I analyzed in [You Are the Bottleneck](../you-are-the-bottleneck/index.md): when arrival rate exceeds your service rate, the queue grows and your quality of judgment degrades before your throughput does. **Reviewing finished work is cheaper than steering work in progress, and it is the only kind of attention that makes the next run better.** ## What to Do Next - Take the last five tasks where an agent interrupted you and classify each interruption: what-does-done-look-like, what-is-true, what-would-you-do. - Fix the most common class first: sharper acceptance criteria, a new automated check, or a new rule in your standing documents. - Before your next handoff, write the definition of done as commands someone else could run, not as a description of what you want. - Split any task longer than the model's reliable horizon into units that are each verifiable and reversible. - Add one standing stop condition to every handoff: blocked means stop and report, never improvise. - Set a fixed review time for finished runs, and fold every failure back into the specification template. ## See also - [The Acceptance Gap](../the-acceptance-gap/index.md) - why "the model says done" is not done, and the two different gaps that checkers and reactions close. - [Verifying Code Without Reading It](../verifying-code-without-reading-it/index.md) - the concrete checker stack that replaces human eyes on every diff. - [My AI Workflow](../my-ai-workflow/index.md) - the skills and standing documents that let agents reuse my judgment instead of asking for it. - [Managing Many Concurrent LLM Agent Sessions](../managing-many-llm-agent-sessions/index.md) - the cognitive cost of live supervision, the problem this workflow answers. - [Scaling Yourself Horizontally](../scaling-yourself-horizontally/index.md) - agents as nodes that act in your absence, and which decisions belong in a system. - [Zero Touch Engineering](../zero-touch-engineering/index.md) - the fully closed loop these workflow changes point toward. - [The Shifting Bottleneck](../the-shifting-bottleneck/index.md) - where the constraint moves once delegation actually works. ## References - [METR, "Measuring AI Ability to Complete Long Tasks"](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) - source for the one-hour 50% horizon, the seven-month doubling time, and the human comparison. - [Anthropic, "Building Effective Agents"](https://www.anthropic.com/research/building-effective-agents) - grounding for environment feedback loops and the workflow-versus-agent distinction. - [Wikipedia, "Test oracle"](https://en.wikipedia.org/wiki/Oracle_(software_testing)) - the oracle problem: without an independent check, plausible output cannot be told from correct output.