Building in the Agentic Era, Part 3.
I ran one feature through my spec pipeline with a single rule: the agent was not allowed to make anything up. It stopped and asked 89 questions. Only one was obvious. 78 of them changed the spec.
Those 78 were decisions nobody had made yet. Without that rule, the agent would have made all of them itself, quietly, in a document that looked finished.
That risk grows once agents start reading your specs. When agents write the code, the spec becomes the thing you manage. Most teams still treat specs like long prompts: written once, skimmed, and out of date by the second sprint. That works for a demo. It breaks the first time an agent builds, with total confidence, from a document that stopped being true three weeks ago.
In Issue 04 I wrote about comprehension debt, the gap between what a system does and what people understand about it. Specs are where much of that understanding is supposed to live. When they drift, the debt grows, and the agents keep building on top of it.
This week is practical: what the research says, six rules I now use for any document an agent builds from, a checklist you can use tomorrow, and a playbook kit with ten prompts you can paste into your agent.
New here? Two engineering leaders on building and leading with AI inside a large enterprise. Every Tuesday and Thursday, free, in your inbox. Subscribe free
More context has a cost
The first instinct is to give the agent more. More documents, more rules, a bigger instructions file. The research says be careful.
A study published on arXiv this year, Evaluating AGENTS.md, tested repository context files, the AGENTS.md-style instruction files a lot of teams now keep for their coding agents. Across several models and agents, those files did not generally improve task success, and they raised inference cost by more than 20% on average. The agents followed the instructions fine. Broad overviews of the codebase just didn’t help.
Birgitta Böckeler at Thoughtworks found the opposite problem when she tried the popular spec-driven tools. One, Kiro, turned a small bug into four user stories and 16 acceptance criteria. Of another, Spec Kit, she wrote: “I’d rather review code than all these markdown files.”
That gives you two ways to fail. With too little structure, the agent guesses. With too much, nobody reads it, and either way you end up with documents nobody trusts.
Spec drift does the damage
Böckeler describes three levels. Spec-first: write a spec for the task, hand it to the agent. Spec-anchored: keep the spec and update it with every change. Spec-as-source: people edit only the spec and never touch the code.
Most teams are spec-first, and that’s fine for small work. The trouble starts when a spec-first document gets treated like a living one. Someone points the agent at last quarter’s PRD (product requirements document), and it builds faithfully from something that’s no longer true.
That’s spec drift. Inside a large enterprise, anything that matters needs to be spec-anchored, and it’s the hardest level to hold. Keeping documents current is work with no alarm when it gets skipped. Nothing breaks the day a spec goes stale. It breaks later, somewhere else.
What I built to test this
On my own time, I built an open-source Claude Code plugin called Spec Factory. It takes a rough idea, a ticket, or a half-written PRD and turns it into a reviewed, traceable set of specs a coding agent can build from: an idea brief, a PRD, a UX design when there’s a new screen, a technical design with its key decisions, and feature specs. It writes specs only. No product code.
I’m not pitching a tool. GitHub’s Spec Kit and Kiro are solid, and they’re a big reason I started thinking about this. I built my own to answer one question: what would it take for me to trust a document an agent helped write? Here’s where I landed. Every rule below works without my tool.
Six rules for documents agents build from
1. If you can’t tell whether a spec is current in ten seconds, neither can your agent. On most teams, “done” lives in someone’s head or in a field nobody updates. Spec Factory computes status from what’s on disk: draft, reviewed, approved, or stale.
2. Agents draft. Humans sign. Agents can draft, review, and critique. They never approve. I enforce that with a separate approval step agents can’t write to. It’s the same idea from Issue 02: the human hours move to the plan and the outcome. Approving the plan is one of those hours.
3. Approved means frozen. Once a document is approved, nobody edits it in place. Changes go through a proposal that a person approves. Then everything downstream gets flagged. Change a requirement in the PRD, and the technical design and feature specs that depend on it show as stale until someone looks. That ripple is the piece most teams are missing.
4. Orphans are where scope creep hides. Every requirement should land in at least one spec. Every spec should point back to a requirement or a decision. Anything that doesn’t trace in both directions is either missing work or extra work.
5. A skipped check that shows green is worse than no check. Format and traceability checks cost almost nothing, so they run before any agent reviews anything. Next comes a panel of fresh reviewers with no memory of the drafting session, then a second model. Every check reports pass, fail, or why it didn’t run.
6. Make the agent ask instead of guess. If the owner didn’t provide a number or a constraint, the agent asks, and every open question is recorded with where it came from. This is where the 89 questions came from. Agent-written documents rarely fail on structure. They fail on confident, reasonable-sounding details that nobody decided.
Isn’t this just waterfall?
Fair pushback, and it can be. The Thoughtworks podcast on this topic warns about teams spending days polishing specs before anyone builds anything. Two things keep me out of that trap.
First, size the process to the work. Spec Factory has three weights (side project, standard, and enterprise), and the depth of the interview scales with each. A bug fix should never get the full chain. Second, the loop is short now. When an agent can rebuild from a revised spec in minutes, a wrong spec costs a lot less than it used to. Plan less, but keep what you plan true.
What I don’t know yet
My results are thin, and I want to be upfront about it. The 89 questions came from a simulated run on one standard-sized feature, with a subagent playing the product owner from a script. It ended with 18 documents approved and zero format failures at handoff. That proves the machinery works. It doesn’t prove a real team ships faster.
The guardrails that stop an agent from approving its own work catch mistakes. They are not a security boundary. And more documents means more to read, which is exactly Böckeler’s warning. I’m still working on that one.
If you try one thing this week: pick the last spec your agents built from and ask two questions. Is it still true? Who approved it? If you can’t answer both in a minute, you’ve found your gap.
A checklist you can steal
Copy it, change it, make it yours. The playbook kit below turns each line into a prompt your agent can run.
Before the agent builds:
Every document it will read has a clear status and an owner.
A person, not an agent, approved it.
The process fits the risk. Small fix, small spec.
When something changes:
Change through a proposal, not an edit in place.
Flag every downstream document that depends on what changed.
Before you call it done:
Every requirement traces to a spec, and every spec traces back.
Every skipped check is visible, with a reason.
Every number and constraint came from a person, not the agent.
Every month:
Prune your agent instruction files. If the agent could learn it by reading the code, cut it.
The playbook kit
Download the kit below. It has the spec health check and ten prompts you can paste into whatever agent you already use: Claude Code, Codex, Copilot, Cursor, Kiro or a plain chat. They set up the rules, interview you before you write a spec, check before you build, and catch drift. Prompts 9 and 10 turn the rules into hard stops if you want them. No scripts, no new tools. Save a copy and run prompts 1 and 2 on one repo this week.
If you want to go deeper
Understanding Spec-Driven Development by Birgitta Böckeler, where the three levels come from.
What is spec-driven development? from the Thoughtworks Technology Podcast, on the risks and the waterfall question.
Evaluating AGENTS.md on arXiv, the study behind the 20% cost finding.
GitHub Spec Kit, GitHub’s open-source toolkit for spec-driven development.
Spec Factory, if you want to look under the hood. Open source, MIT licensed.
One ask: forward this to the person on your team who owns the specs your agents build from. If nobody comes to mind, that’s worth a conversation too.
I’m curious how other teams handle this. How do you keep specs true once agents start building from them? Hit reply or leave a comment.
Next Tuesday, Rishi is back.
The opinions expressed here are our own and do not represent any employer, past or present. Nothing in this publication describes any specific company’s internal systems, data, vendors, teams, metrics, or plans. Sponsored placements and sponsored briefs are clearly labeled.
Till next time,