BDD and acceptance evidence for coding agents

Using behavioural examples and evidence to decide whether a specification is ready for the next commitment.

An export can contain exactly the specified columns and still be unusable for reconciliation. This is the distinction between verification and validation: verification examines conformance to the applicable contract; validation examines whether the requirements and delivered behaviour serve the intended need. An activity may inform both, but success in one does not settle the other.

There is another dependency beneath all this checking: the expected result. The test oracle problem concerns how to determine what the correct result should be. A second agent handed the same mistaken premise may faithfully reproduce it.

An independent basis must support or challenge the disputed expectation without assuming it is correct. Seek it in domain examples grounded in observed work, an applicable external contract, a justified reference or a separately reasoned property. A property derived only from a disputed rule can assess conformance to that rule; it cannot justify the rule itself. Examples explain interpretation, properties generalise obligations and experiments challenge assumptions. Choose the combination that addresses the risk.

Use BDD to discover and express acceptance criteria

“Export the filtered invoices” can sound settled until finance expects a snapshot and the developer expects whatever the database returns when the worker runs. A concrete example makes that disagreement visible: an invoice changes after the request is accepted but before the file is assembled. Which value belongs in the file? The answer changes the contract and may change the design.

Behaviour-driven development gives that discussion a place in development. It predates coding agents: Dan North’s original account connects TDD with analysis and acceptance testing through behaviour and shared language, and recognises that a failing test may expose an outdated premise. Cucumber describes the practice as an iterative combination of discovery, formulation and automation, applied to small upcoming changes and revisited as questions arise.

BDD is optional. Where it helps, its examples can supply much of a specification’s behavioural content without adding another document layer. North’s and Cucumber’s accounts describe practices and conventions; they do not establish an effect size for introducing BDD into agent-assisted development. The choice is whether the examples improve the decision the team needs to make.

Let concrete cases expose disagreement

Example Mapping separates rules, examples, questions and deferred scope. The disputed invoice value becomes an explicit question to resolve before it can become an acceptance example. Agreement on the example records the current decision; it does not make the decision true without the domain basis described above.

That requires perspectives capable of challenging one another. Domain expertise explains the need, development examines feasibility and boundaries, and testing probes examples and omissions. These are contributions to continuing collaboration; they do not require exactly three people or a new approval board.

An agent can propose edge cases, expose inconsistent wording and retrieve repository evidence. Assigning agents the personas of business representative, developer and tester does not, by itself, supply independent evidence of user intent. On a solo project, observations and domain sources still have to challenge the author’s assumptions; simulated agreement cannot do that work for them.

Discuss detailed scenarios close to the relevant implementation work, so their assumptions can be checked promptly. Cucumber recommends discovery workshops near the start of the relevant story’s development. Project-level risks and expensive feasibility questions may need attention earlier. Treating the entire product’s scenarios as settled before building anything would preserve the waterfall problem, however readable the notation.

Make scenarios part of the contract

Agreed BDD scenarios belong among the acceptance criteria: they state concrete situations and the outcomes required for acceptance. Keep the general rule with them. Otherwise a collection of examples leaves the implementer to infer what happens in every unlisted case.

Scroll horizontally to see all columns.

Participants discuss and agree the examples, record them as scenarios and connect them to the system through test code. An automated scenario can therefore be both a readable acceptance criterion and an executable check. Not every acceptance criterion or test needs Given/When/Then.

Give each rule and scenario one maintained home. If a feature file owns an example, reference it from the specification’s entry point. Refactoring test code must preserve the agreed outcome. If a scenario misstates a rule or the requirement itself needs revision, use the decision process in Part 4 to correct the affected expectations.

Choose the notation for the people who will use it

Gherkin provides a scenario notation, and Cucumber provides tools that connect scenarios to executable checks. Collaborative examples and ordinary tests can serve the same development practice without a Gherkin runner. In a Go library, domain-reviewed cases may become table-driven tests. Add the translation layer when the shared executable representation helps people review and maintain behaviour; if its only readers already work comfortably in test code, it may add little.

Where Gherkin is useful, describe observable outcomes and keep replaceable mechanics in the automation. The Gherkin reference distinguishes a business rule from the scenarios illustrating it and directs assertions towards observable results. An unchanged domain rule should usually survive a move from a web interface to an API. Browser interactions belong in the scenario when the interaction itself is required; elsewhere, selectors and clicks can conceal the obligation being tested.

A revoked download request

The following scenario is a hypothetical acceptance example for a permission boundary under investigation. It concerns a new request after access has been revoked. It does not decide what happens to a transfer that was already authorised, and it does not establish that the rule is feasible or that the whole export is ready for release. The complete export rules and investigation remain in Part 5’s export walkthrough.

Feature: Invoice export downloads

  Rule: Each download request requires current account authorisation

    Scenario: Access is revoked after an export is prepared
      Given an administrator has a completed export for account "A"
      And the administrator no longer has access to account "A"
      When the administrator starts a new download request for that export
      Then the download is denied
      And no invoice data is returned

The rule applies at the download boundary, including a direct storage URL if the design uses one. A queue, object store, helper function or database layout remains a design choice.

Keep the examples alive during implementation

Automate selected examples at the boundary they concern, and establish that the checks detect the relevant violation. A discovered case may send the work back to the domain rule; an agreed case can remain as regression evidence. TDD supplies a smaller loop of failing test, implementation and refactoring within this work. It is feedback inside an authorised increment, not a substitute for deciding whether the increment is ready.

Adding Cucumber after implementation may produce useful regression tests, but examples guide development only when they participate in its decisions. If the contract is not ready, implementation feedback should return the question for investigation or revision rather than silently settle it.

Assess the spec and decide what evidence acceptance needs

Choose evidence while agreeing the contract, when an obligation can still be examined rather than merely handed to an implementer:

Scroll horizontally to see all columns.

For the revoked-download scenario, establish revoked access through the relevant permission mechanism, request the actual download boundary and check that no protected data is returned. A fake that always denies access would merely demonstrate that the fake denies access. The criterion states the expected outcome; the executed check must encounter the state change that could violate it.

For suitability, examine user and operational evidence: can finance complete its reconciliation with the exported data? Usability and accessibility may need both checks against stated criteria and observation of actual use. Those observations can challenge the criteria themselves. A scenario or component check covers selected executions and interactions; it does not establish every concurrent execution or a system-wide property.

Amazon’s formal-methods experience shows how precise models can expose distributed-system design defects beyond ordinary reviews and tests. The result remains about the represented system and stated properties; establishing correspondence with an implementation is further work.

Is the spec good enough for the next commitment?

Clarity is necessary, but it can make a wrong requirement exceptionally easy to implement. Review the draft against its purpose and evidence:

• Justified: Consequential rules and the intended outcome have a credible domain basis; reviewers can explain why they matter.
• Consistent: Rules agree with one another and with applicable external and shared contracts. Deliberate changes are identified.
• Sufficiently complete: The increment’s important outcomes, failure effects and boundaries are covered, with remaining questions explicit.
• Feasible: Critical assumptions have supporting evidence, or investigating them is the bounded next assignment.
• Verifiable: Acceptable and unacceptable outcomes can be distinguished, and suitable evidence can be obtained.

The review should end in a concrete choice: implement the bounded scope, investigate the uncertainty controlling it, or revise the contract. That choice is about the next commitment, not a verdict on every future capability.

An unresolved point blocks dependent implementation when different answers could change a promised outcome, break a consumer, violate a shared invariant or invalidate the work. Resolve it or narrow the assignment. Independent later capabilities can remain open, as can private implementation choices that preserve the obligations. When the decision is to implement, Part 4 supplies the agent and environment readiness checks; when it is to investigate or revise, preserve the question, evidence and owner so the next assignment cannot turn silence into authority.