Writing specifications for coding agents

Two implementations of “remove duplicate contacts” can look reasonable and return different records: one keeps the first occurrence, the other the last. Choosing between them changes what callers receive. The same request leaves identity, ordering and malformed-record handling undecided; a plausible implementation cannot settle those rules on the product owner’s behalf.

A bounded specification makes those decisions visible. It states the outcome, the settled rules that determine it, the constraints that must hold and the choices that remain with the implementer. Questions that are still open belong in the working task, with their resolution preserved in history; silence in the current contract is not permission. The result is a contract an agent can retrieve and work against; its rules still need the domain basis and evidence that justify the expected outcome.

Write a bounded specification for the agent

A useful specification distinguishes acceptable results from plausible mistakes. An agent also needs to retrieve the current decisions, tell obligations from suggestions and recognise questions it has no basis to answer. These demands favour precision about consequential behaviour within a bounded scope. They say little about document length: pages of happy-path description can still omit the one rule that determines whether the implementation is acceptable.

Start compactly, then justify additions

A starting structure can be as small as this:

# Capability or change
Scope/version and responsible decision-maker

## Purpose and scope
Outcome, included change, non-goals and promises to preserve

## Requirements
CAP-01: A settled, independently assessable outcome with its terms, inputs/outputs, invariants and failure behaviour
CAP-02: Another independently assessable outcome owned by the capability
References to binding interfaces and constraints

## Acceptance
Decisive examples/properties, the independent basis for expected outcomes, and verification

Adapt it to the repository. These are information roles, not five required files. Each rule should describe an independently assessable outcome with one authoritative home. One capability owner may own several outcomes. GitHub Spec Kit’s template also includes user scenarios, functional requirements and measurable success criteria; that makes it an example of a convention, not evidence that every section improves every assignment.

An identifier such as CAP-01 gives a test, review or change discussion a stable reference to a rule. When that rule changes, its references help locate the examples and checks to review. Use IDs when those connections are useful; a descriptive heading or link may suffice for a small local change. The identifier itself supplies no evidence that the requirement is right or adequately tested.

Keep an ID when moving or reordering its rule, and do not reuse a retired ID for an unrelated obligation. The ID identifies the rule; a contract revision or repository commit identifies the wording a check assessed. Questions that block a future commitment stay in the task record until resolved; history records superseded decisions without making them current requirements.

A paragraph earns its place by enabling a decision, preventing an important misunderstanding or making evidence accessible. Before removing it, check whether it contains a constraint, rationale or exception that would be difficult to recover.

Include what determines the result

For a local correction, the existing contract may supply most of what follows. Add or revise only the information needed to decide and assess the change.

Start with the outcome: who or what benefits, which problem the change addresses and how success will be observed. State the behaviour to add or change, relevant non-goals and existing promises to preserve. Retain the rationale that explains those decisions.

Where interpretation can change the result, define domain terms such as “duplicate”, “active”, “latest”, “owner” and “completed”, and use them consistently. Specify the relevant inputs and outputs: accepted values, units, identity, required fields, defaults and observable results. Include schemas or signatures when a component or consumer depends on them.

Behavioural rules describe preconditions, state changes, postconditions and invariants. Examine the boundaries and failures that could affect acceptance: empty, invalid or oversized inputs; partial failure; permissions; retries; ordering; concurrency; time and lifecycle.

Negative obligations deserve equal attention. “Exporting must not mutate invoices” restricts behaviour; “scheduled exports are outside this increment” restricts scope. Likewise, a rejected batch leaving no partial writes may be central to correctness. Folding all these statements into a list of non-goals would lose the distinction between work that is absent and behaviour that is forbidden.

Binding constraints include compatibility promises, supported environments, mandated dependencies and architectural boundaries. Give their reasons and sources. Where latency, capacity, resource use, accessibility or operational behaviour matters, state the obligation, its conditions and how it will be assessed. “Fast” and “robust” cannot decide acceptance on their own.

Consider “complete within two seconds”. To make it a dependable performance requirement, specify the operation, workload, environment and measurement convention. If those remain unknown, commission the measurement or mark the target as provisional. A precise number cannot compensate for unspecified conditions.

Choose acceptance evidence from agreed scenarios, representative examples, boundary cases, counterexamples and properties. Link to verification that can assess a rule without taking its expected result from the proposed implementation. Keep settled requirements in the current contract; record provisional assumptions and open questions in the working task with an owner, the evidence needed and the work that depends on their resolution.

NASA’s requirements guidance supplies established checks for clarity, consistency, rationale, traceability and verifiability. Use those checks selectively; adopting a checklist does not require adopting the whole systems-engineering process. Its scope and the related context guidance are recorded in Appendix A.

Write the condition and the required response

“The system shall handle invalid contacts correctly” leaves the implementer to choose between rejecting the input, skipping invalid records and returning a partial result. Before improving the sentence, settle that choice. In the hypothetical contact example below, any invalid ID causes the whole operation to fail.

Write the requirement around the condition that matters, the capability responsible and the result a caller can observe. Naming the operation identifies who owes that behaviour without prescribing a class or algorithm.

EARS (Easy Approach to Requirements Syntax) supplies sentence patterns for expressing those decisions. Three fit the contact contract:

Scroll horizontally to see all columns.

Choose the form by when the obligation applies. When introduces a triggering event; While makes an obligation apply for as long as a stated condition holds. Checking permission when a download starts and requiring permission throughout the transfer are different promises. The EARS guide also covers optional features and combinations of patterns; the contact operation does not need every form.

Keep definitions and scope beside the requirements to establish their meaning and applicability. A requirement may need several effects to describe one outcome: rejection with an error and no partial result belong together.

Give supporting material a clear role

An agent may need design notes, repository guidance and a task plan as well as the specification. Keep their roles clear and their relevant contents accessible. A suggested approach can otherwise become indistinguishable from a binding constraint, while a real constraint can disappear because it lives in another file.

Imagined future features remain hypotheses or backlog items until evidence and dependencies justify commitment. A guessed product decision belongs in the working record with the uncertainty and decision still needed. Repeating a generated assumption must not promote it into a settled rule.

Design notes can propose private classes, helpers, file layouts or algorithms; those choices can also be left to implementation. An approach becomes binding only when a real constraint or justified design decision requires it, with the reason recorded. A task plan can carry the coding sequence; required domain protocols and state transitions belong in the contract.

Working instructions and repository conventions belong in the task brief or repository guidance. The current contract should link to its decision history, with rejected and superseded alternatives clearly marked. Code references should locate the relevant boundary and explain surprising facts and preservation requirements without duplicating a directory inventory. For other references, give the agent precise paths, sections or versions, explain what each governs and check that it can retrieve them. Resolve conflicts between current sources with the responsible decision-maker before dependent work proceeds.

Test code owns setup, selectors and adapters. Agreed expected behaviour belongs in the specification or its canonical executable examples.

A public API’s error codes, a message schema, memory-ownership rules or a mandated numerical algorithm may define correctness. Whether a returned buffer survives the next iterator call is an observable contract; how a private buffer is allocated is usually a design choice. Both are technical, but they impose different obligations on callers and implementers.

The same distinction matters when work is divided among implementers. Shared interfaces and interaction invariants need agreement before dependent implementation. Private methods do not all acquire that status merely because several agents are working on the project. If an agreed design document governs the work, make it available to the agent: a constraint the agent cannot retrieve is an inadequate basis for delegation.

Separate precision, coverage and prescription

“More detail” can mean three different changes. They should be assessed separately.

Scroll horizontally to see all columns.

A narrow feature can have precise behaviour and substantial implementation freedom. That combination is useful: the implementer knows what must survive a design change without having to preserve every detail of the first design.

Two questions help test the boundary. Could two competent implementations satisfy the text yet disagree on an outcome a user or dependent component cares about? Clarify that outcome or explicitly delegate the choice. Could a substantially different implementation preserve every promise? Examine whether the prescribed mechanism is necessary.

These questions also expose a common ambiguity in the specification itself. Ordering that is explicitly unconstrained grants discretion; omitted ordering may simply be an oversight. An unresolved product question does not grant the same freedom as choosing a private data structure. State the distinction wherever it affects dependent work.

Add detail where a wrong guess would cause significant rework, break a consumer or invalidate the increment. If the answer lacks evidence, investigate it. Expanding the prose cannot settle what the author does not yet know.

Precision in a small contract

“Remove duplicate contacts” leaves several consequential choices unanswered: identity, retained record, output order, mutation and invalid-input behaviour. Naming a hash map in ContactDeduplicator settles a mechanism while leaving those choices open.

The following hypothetical contract makes the decisions explicit. They are invented for this example; a real system would need domain agreement before adopting them.

Purpose: Collapse repeated entries before displaying an imported contact list.

Input: A finite ordered sequence of records. Other fields are opaque to this operation; this is domain scope, not a separate requirement.

DEDUP-01: The contact deduplication operation shall identify records as duplicates if and only if their contact_id strings are equal under exact case-sensitive comparison.

DEDUP-02: When given input in which every record has a non-empty string contact_id, the contact deduplication operation shall return the first record for each ID with all its fields unchanged and in the records’ original relative order.

DEDUP-03: The contact deduplication operation shall not mutate the input on success or rejection.

DEDUP-04: If any input record has a missing, empty or non-string contact_id, then the contact deduplication operation shall reject the entire input with an invalid-input error and return no partial result.

DEDUP-05: When given empty input, the contact deduplication operation shall succeed with an empty result.

Scope: Matching by email, merging fields and persistent writes are non-goals. The algorithm is an implementation choice.

The acceptance examples now distinguish implementations that the original request would have left indistinguishable:

Scroll horizontally to see all columns.

Here, a/Ada abbreviates an ID and another field; it specifies no wire format. The runtime signature and error representation come from the enclosing API contract. If there is no such contract, agree them before integration.

The case with a valid record followed by an invalid one checks more than whether an error occurs. Skipping the invalid record or returning the valid prefix would violate DEDUP-04. Mutating the input before rejecting it would violate DEDUP-03, even if the operation returned the right error and no result. One case can expose failures against several requirements.

Two testers can derive different valid checks from this contract. One checks that records with IDs a, a collapse to one; another checks that a, A both remain. The first check would also pass an implementation that incorrectly ignores case. The second rejects that mistake. Their different inputs expose different faults; they need not disagree about the requirement.

If the testers instead expect different results for the same a, A input, examine the rule. Here, DEDUP-01 explicitly requires case-sensitive comparison, so expecting one record is a mistaken expectation. Under the original “remove duplicate contacts” request, that distinction was unresolved. Writing an assertion would silently decide it. Part 3 explains how to connect agreed examples to checks and assess the evidence they provide.

The requirements also support broader checks: output IDs are unique under the stated comparison, retained records match their first occurrence, and applying the operation twice preserves the result. Those obligations remain the same whichever algorithm implements them. Tests should exercise the obligations without requiring a hash map.

The example is now precise enough to implement, but precision does not supply its own justification. The examples and properties above can show whether an implementation conforms to first-record retention and case-sensitive identity; they cannot show that those are the outcomes the importing workflow needs. That question remains open until an independent basis supports or challenges the expectation. Part 3 follows that question through agreement, examples and evidence before acceptance.

Appendix A. Evidence and limitations

The recommendations in this article are engineering judgements, not a complete method validated by the studies below. The evidence here concerns targeted clarification and supplemental repository guidance. It establishes no universal specification length, mandatory template or productivity gain. The examples are hypothetical; no development experiment was run for this article.

Does clarification help?

ClarifyGPT asks whether targeted questions improve code generation. Orchid examines what happens when requirements still permit different interpretations.

ClarifyGPT, FSE 2024

ClarifyGPT detects ambiguity, asks targeted questions and supplies the clarified requirement before code generation. In a study with ten participants, Pass@1 on MBPP-sanitized was 70.96% for default GPT-4 and 80.80% for ClarifyGPT using their answers. Pass@1 estimates how often a single generated solution passes the supplied tests. MBPP-sanitized is a hand-verified subset of Mostly Basic Python Problems, a benchmark of short Python programming tasks.

The result supports targeted clarification for the evaluated tasks. It does not establish whether an answer reflects the right product decision. This is peer-reviewed function-generation research; the larger automated evaluation used simulated responses. It does not validate a whole-project workflow or an optimum specification format.

Orchid ambiguity study, April 2026 preprint

Orchid compared clear and deliberately ambiguous requirements across 1,304 function tasks and 5,216 variants. The paper reports an average Pass@1 decline of 7.22 percentage points under ambiguity, with effects varying by model and ambiguity type. On the HumanEval-derived subset, GPT-4’s measured share of response pairs producing different outputs for the same input rose from 14.09% to 28.29%. On the BigCodeBench subset (Orchid-BCB), CodeLlama’s conflict rate fell from 50.12% to 47.62%. The authors caution that its baseline Pass@1 of 6.22% leaves too few correct outputs for a meaningful consistency comparison.

The practical concern is that plausible interpretations can produce different results. Examples and questions can help expose which interpretation is intended. However, the ambiguity was synthetic, and the evaluated model versions limit transfer to repository agents. Clear wording still cannot establish whether the intended requirement is right.

Does more context help?

Supplemental repository guidance and a specification’s presentation history are different questions. This section covers repository guidance; Part 4 covers specification history and changing instructions. Neither study establishes that more context is always better.

AGENTS.md evaluation, February 2026 preprint

This study examined whether repository instruction files help coding agents resolve issues. It evaluated SWE-bench Lite, with 300 tasks from 11 Python repositories, and AGENTbench, with 138 tasks from 12 additional Python repositories. Generated instructions reduced task success in five of eight agent/benchmark settings and raised average execution cost by 20% on SWE-bench Lite and 23% on AGENTbench. Developer-written files improved success for three of four agents on AGENTbench while also increasing cost.

The finding motivates scrutiny of redundant instructions and their cost. It does not justify omitting necessary requirements. The study concerns supplemental repository guidance, not feature specifications or specification length. Some task descriptions and tests used LLM assistance, and effects varied across agents.

What the other sources contribute

NASA’s requirements guidance informs the checks for clarity, consistency, rationale, traceability and verifiability. Anthropic’s context guidance argues for relevant information and a balance between vague direction and brittle instruction recipes. Neither supplies an experimentally established token budget for a specification.

Boeckeler’s first-hand experiments reported an agent recreating classes described in research notes about an existing system. The incident illustrates the distinction between describing what exists and requesting a change. It is a historical observation, not evidence that current tool versions necessarily behave the same way.

These sources constrain how much can be claimed here. The studies can motivate questions about ambiguity and context; the guidance can supply checks and cautions; the incident can illustrate a role confusion. None makes the contract’s chosen outcomes valid by itself. Check the relevant tool documentation before adopting a workflow, because it may have changed since the article was written.