Writing specifications for coding agents

Writing specifications that give coding agents room to choose without leaving them to choose the behaviour.

“Remove duplicate contacts” sounds like a small request. It leaves open which fields identify a contact, which record survives, whether order matters and what happens when one record is malformed. An agent can fill those gaps with a plausible implementation. That implementation then becomes behaviour for callers and users, even though nobody deliberately chose it.

A bounded specification makes those decisions visible. It states the outcome, the rules that determine it, the constraints that must hold and the choices that remain with the implementer. It also records questions that are still open instead of turning silence into permission. The result is a contract an agent can retrieve and work against. It is not a certificate that the requirement is right; the next article covers the agreement and evidence needed to challenge or support the expected outcome.

Write a bounded specification for the agent

A useful specification distinguishes acceptable results from plausible mistakes. An agent also needs to retrieve the current decisions, tell obligations from suggestions and recognise questions it has no basis to answer. These demands favour precision about consequential behaviour within a bounded scope. They say little about document length: pages of happy-path description can still omit the one rule that determines whether the implementation is acceptable.

Start compactly, then justify additions

A starting structure can be as small as this:

# Capability or change
Status, scope/version and responsible decision-maker

## Purpose and scope
Outcome, included change, non-goals and promises to preserve

## Contract
Terms, applicable inputs/outputs, rules, invariants and failure behaviour
References to binding interfaces and constraints

## Acceptance
Decisive examples/properties, an independent basis for expected outcomes, and verification

## Open decisions
Assumptions and questions; owner, evidence needed, dependent work and resolution condition for each consequential unknown

Adapt it to the repository. These are information roles, not five required files. GitHub Spec Kit’s template also includes user scenarios, functional requirements and measurable success criteria; that makes it an example of a convention, not evidence that every section improves every assignment.

A paragraph earns its place by enabling a decision, preventing an important misunderstanding or making evidence accessible. Before removing it, check whether it contains a constraint, rationale or exception that would be difficult to recover.

Include what determines the result

For a local correction, the existing contract may supply most of what follows. Add or revise only the information needed to decide and assess the change.

Start with the outcome: who or what benefits, which problem the change addresses and how success will be observed. State the behaviour to add or change, relevant non-goals and existing promises to preserve. Retain the rationale that explains those decisions.

Where interpretation can change the result, define domain terms such as “duplicate”, “active”, “latest”, “owner” and “completed”, and use them consistently. Specify the relevant inputs and outputs: accepted values, units, identity, required fields, defaults and observable results. Include schemas or signatures when a component or consumer depends on them.

Behavioural rules describe preconditions, state changes, postconditions and invariants. Examine the boundaries and failures that could affect acceptance: empty, invalid or oversized inputs; partial failure; permissions; retries; ordering; concurrency; time and lifecycle. Use stable identifiers where they help connect rules to decisions and evidence.

Negative obligations deserve equal attention. “Exporting must not mutate invoices” restricts behaviour; “scheduled exports are outside this increment” restricts scope. Likewise, a rejected batch leaving no partial writes may be central to correctness. Folding all these statements into a list of non-goals would lose the distinction between work that is absent and behaviour that is forbidden.

Binding constraints include compatibility promises, supported environments, mandated dependencies and architectural boundaries. Give their reasons and sources. Where latency, capacity, resource use, accessibility or operational behaviour matters, state the obligation, its conditions and how it will be assessed. “Fast” and “robust” cannot decide acceptance on their own.

Consider “complete within two seconds”. To make it a dependable performance requirement, specify the operation, workload, environment and measurement convention. If those remain unknown, commission the measurement or mark the target as provisional. A precise number cannot compensate for unspecified conditions.

Choose acceptance evidence from agreed scenarios, representative examples, boundary cases, counterexamples and properties. Link to verification that can assess a rule without taking its expected result from the proposed implementation. Keep accepted rules, provisional assumptions and open questions distinguishable; for consequential unknowns, identify who or what can resolve them and which work depends on the answer.

NASA’s requirements guidance supplies established checks for clarity, consistency, rationale, traceability and verifiability. Use those checks selectively; adopting a checklist does not require adopting the whole systems-engineering process. Its scope and the related context guidance are recorded in Appendix A.

Give supporting material a clear role

An agent may need design notes, repository guidance and a task plan as well as the specification. Keep their roles clear and their relevant contents accessible. A suggested approach can otherwise become indistinguishable from a binding constraint, while a real constraint can disappear because it lives in another file.

Imagined future features remain hypotheses or backlog items until evidence and dependencies justify commitment. A guessed product decision needs an explicit record of the uncertainty and the decision still needed. Repeating a generated assumption must not promote it into an accepted rule.

Design notes can propose private classes, helpers, file layouts or algorithms; those choices can also be left to implementation. An approach becomes binding only when a real constraint or justified design decision requires it, with the reason recorded. A task plan can carry the coding sequence; required domain protocols and state transitions belong in the contract.

Working instructions and repository conventions belong in the task brief or repository guidance. The current contract should link to its decision history, with rejected and superseded alternatives clearly marked. Code references should locate the relevant boundary and explain surprising facts and preservation requirements without duplicating a directory inventory. For other references, give the agent precise paths, sections or versions, explain what each governs and check that it can retrieve them. Resolve conflicts between current sources with the responsible decision-maker before dependent work proceeds.

Test code owns setup, selectors and adapters. Agreed expected behaviour belongs in the specification or its canonical executable examples.

A public API’s error codes, a message schema, memory-ownership rules or a mandated numerical algorithm may define correctness. Whether a returned buffer survives the next iterator call is an observable contract; how a private buffer is allocated is usually a design choice. Both are technical, but they impose different obligations on callers and implementers.

The same distinction matters when work is divided among implementers. Shared interfaces and interaction invariants need agreement before dependent implementation. Private methods do not all acquire that status merely because several agents are working on the project. If an agreed design document governs the work, make it available to the agent: a constraint the agent cannot retrieve is an inadequate basis for delegation.

Separate precision, coverage and prescription

“More detail” can mean three different changes. They should be assessed separately.

Scroll horizontally to see all columns.

A narrow feature can have precise behaviour and substantial implementation freedom. That combination is useful: the implementer knows what must survive a design change without having to preserve every detail of the first design.

Two questions help test the boundary. Could two competent implementations satisfy the text yet disagree on an outcome a user or dependent component cares about? Clarify that outcome or explicitly delegate the choice. Could a substantially different implementation preserve every promise? Examine whether the prescribed mechanism is necessary.

These questions also expose a common ambiguity in the specification itself. Ordering that is explicitly unconstrained grants discretion; omitted ordering may simply be an oversight. An unresolved product question does not grant the same freedom as choosing a private data structure. State the distinction wherever it affects dependent work.

Add detail where a wrong guess would cause significant rework, break a consumer or invalidate the increment. If the answer lacks evidence, investigate it. Expanding the prose cannot settle what the author does not yet know.

Precision in a small contract

“Remove duplicate contacts” leaves four consequential choices unanswered: identity, retained record, output order and invalid-input behaviour. Naming a hash map in ContactDeduplicator settles a mechanism while leaving every one of those choices open.

The following hypothetical contract makes the decisions explicit. They are invented for this example; a real system would need domain agreement before adopting them.

Purpose: Collapse repeated entries before displaying an imported contact list.

Input: A finite ordered sequence of records. The operation validates that every record has a non-empty string contact_id. Other fields are opaque to this operation.

Rules: Equal contact_id strings identify duplicates, using exact case-sensitive comparison. Retain the first record for each ID, with all its fields unchanged. Return retained records in their original relative order. Do not mutate the input.

Failure: If any record lacks a valid contact_id, reject the entire operation with an invalid-input error and return no partial result. Empty input succeeds with an empty result.

Scope: Matching by email, merging fields and persistent writes are excluded. The algorithm is an implementation choice.

The acceptance examples now distinguish implementations that the original request would have left indistinguishable:

Scroll horizontally to see all columns.

Here, a/Ada abbreviates an ID and another field; it specifies no wire format. The runtime signature and error representation come from the enclosing API contract. If there is no such contract, agree them before integration.

The rules also support broader checks: output IDs are unique under the stated comparison, retained records match their first occurrence, and applying the operation twice preserves the result. Those obligations remain the same whichever algorithm implements them. Tests should exercise the obligations without requiring a hash map.

The example is now precise enough to implement, but precision does not supply its own justification. The examples and properties above can show whether an implementation conforms to first-record retention and case-sensitive identity; they cannot show that those are the outcomes the importing workflow needs. That question remains open until an independent basis supports or challenges the expectation. Part 3 follows that question through agreement, examples and evidence before acceptance.

Appendix A. Evidence and limitations

The recommendations in this article are engineering judgements, not a complete method validated by the studies below. The evidence here concerns targeted clarification and supplemental repository guidance. It establishes no universal specification length, mandatory template or productivity gain. The examples are hypothetical; no development experiment was run for this article.

Does clarification help?

ClarifyGPT asks whether targeted questions improve code generation. Orchid examines what happens when requirements still permit different interpretations.

ClarifyGPT, FSE 2024

ClarifyGPT detects ambiguity, asks targeted questions and supplies the clarified requirement before code generation. In a study with ten participants, Pass@1 on MBPP-sanitized was 70.96% for default GPT-4 and 80.80% for ClarifyGPT using their answers. Pass@1 estimates how often a single generated solution passes the supplied tests. MBPP-sanitized is a hand-verified subset of Mostly Basic Python Problems, a benchmark of short Python programming tasks.

The result supports targeted clarification for the evaluated tasks. It does not establish whether an answer reflects the right product decision. This is peer-reviewed function-generation research; the larger automated evaluation used simulated responses. It does not validate a whole-project workflow or an optimum specification format.

Orchid ambiguity study, April 2026 preprint

Orchid compared clear and deliberately ambiguous requirements across 1,304 function tasks and 5,216 variants. The paper reports an average Pass@1 decline of 7.22 percentage points under ambiguity, with effects varying by model and ambiguity type. On the HumanEval-derived subset, GPT-4’s measured share of response pairs producing different outputs for the same input rose from 14.09% to 28.29%. On the BigCodeBench subset (Orchid-BCB), CodeLlama’s conflict rate fell from 50.12% to 47.62%. The authors caution that its baseline Pass@1 of 6.22% leaves too few correct outputs for a meaningful consistency comparison.

The practical concern is that plausible interpretations can produce different results. Examples and questions can help expose which interpretation is intended. However, the ambiguity was synthetic, and the evaluated model versions limit transfer to repository agents. Clear wording still cannot establish whether the intended requirement is right.

Does more context help?

Supplemental repository guidance and a specification’s presentation history are different questions. This section covers repository guidance; Part 4 covers specification history and changing instructions. Neither study establishes that more context is always better.

AGENTS.md evaluation, February 2026 preprint

This study examined whether repository instruction files help coding agents resolve issues. It evaluated SWE-bench Lite, with 300 tasks from 11 Python repositories, and AGENTbench, with 138 tasks from 12 additional Python repositories. Generated instructions reduced task success in five of eight agent/benchmark settings and raised average execution cost by 20% on SWE-bench Lite and 23% on AGENTbench. Developer-written files improved success for three of four agents on AGENTbench while also increasing cost.

The finding motivates scrutiny of redundant instructions and their cost. It does not justify omitting necessary requirements. The study concerns supplemental repository guidance, not feature specifications or specification length. Some task descriptions and tests used LLM assistance, and effects varied across agents.

What the other sources contribute

NASA’s requirements guidance informs the checks for clarity, consistency, rationale, traceability and verifiability. Anthropic’s context guidance argues for relevant information and a balance between vague direction and brittle instruction recipes. Neither supplies an experimentally established token budget for a specification.

Boeckeler’s first-hand experiments reported an agent recreating classes described in research notes about an existing system. The incident illustrates the distinction between describing what exists and requesting a change. It is a historical observation, not evidence that current tool versions necessarily behave the same way.

These sources constrain how much can be claimed here. The studies can motivate questions about ambiguity and context; the guidance can supply checks and cautions; the incident can illustrate a role confusion. None makes the contract’s chosen outcomes valid by itself. Check the relevant tool documentation before adopting a workflow, because it may have changed since the article was written.