Spec tools and hooks: what they automate
Suppose a download test passes, then a shared authorisation helper changes. The requirement and test file remain unchanged. A task list can still say complete. The question for the next reviewer is whether the old result describes the code now being considered for merging.
Part 6 assigns that responsibility to the change review. Tools can make the affected obligations and stale evidence easier to find. They can also generate more material for the reviewer to reconcile. The useful comparison is what each tool does at that boundary, and what work remains.
Kiro already covers much of this workflow
Kiro is a development environment with an integrated specification workflow. Its feature and bugfix specs produce requirements or bug analysis, a design and executable tasks. That is considerably more product functionality than the small checks below.
Its requirement analysis looks across requirements for ambiguity, contradictions and missing cases. Bugfix specs distinguish current, expected and preserved behaviour. In the IDE, property-based testing links generated properties to requirements and exercises the implementation. The documentation explicitly acknowledges that a weak or wrong property can pass while the real behaviour remains wrong.
It would be a poor comparison to describe Kiro as requiring a new specification for every edit. Its maintenance guidance allows ordinary chat fixes, explains how to refine existing specs and synchronise tasks, and distinguishes when fuller bug analysis is useful. The choice is not simply between ceremony and no ceremony. It is which additional preparation or check earns its place in the work.
Compare documented mechanisms at the review boundary
The following are documented mechanisms, not results from a comparative experiment. Documentation was inspected on 21 September 2026. Kiro’s current hook documentation describes IDE 1.x and CLI 3.x; individual features differ by surface. Spec Kit references the inspected convergence procedure, and OpenSpec references its inspected workflow documentation. Those products were not executed for this article; installation, execution and maintenance costs were not compared.
Scroll horizontally to see all columns.
| Case | Existing tools | What the examples below add to the review |
|---|---|---|
| Repair code under an unchanged requirement. | Kiro supports a direct fix or a structured bugfix analysis. Spec Kit’s convergence procedure examines implementation against the feature’s specification, plan and tasks, then appends remediation tasks. | Keep the applicable rule unchanged, follow the affected implementation paths, and distinguish an actual result from a suggested check. |
| Change a shared helper used by other capabilities. | Kiro can analyse requirement interactions and refine related design/tasks. OpenSpec maintains capability specs alongside change deltas. The selected artifacts still determine what a review examines. | A declared relationship map exposes candidate consumers, including requirements whose own files did not change. Missing relationships remain a review risk. |
| Edit code after a passing run. | Task status, a convergence report and an archived change describe different workflow facts. None of those facts, by itself, identifies the inputs of a particular test execution. | An execution record compares the recorded workspace with the current one and reports command success separately from input matching. |
Spec Kit’s convergence procedure takes intent from the feature’s specification, plan and tasks under the project constitution. It inspects current code and appends remediation tasks without rewriting those sources. Missing required artifacts stop the procedure; malformed extension-hook configuration is reported but the procedure continues. Its workflow hooks are distinct from host interception of tool calls.
OpenSpec separates maintained specifications from proposed changes and supports synchronisation and archiving. Its expanded workflow includes verification; verification and archive completion remain separate operations. An archive records lifecycle progress, while conformance still needs evidence for the relevant implementation. Neither this distinction nor a requirement-to-test link is unique to the approach in this series.
The Codex examples implement an inspectable review protocol: select candidate obligations, record what actually ran, and check whether those recorded inputs still match. The capability contract owns the rule; the task record owns the review decision. Kiro’s skills support and command hooks could host similar instructions and checks. This article implements the adapter for Codex; it does not establish that the protocol produces better results than Kiro.
The maintenance work differs too. Kiro integrates requirements, design and task refinement into its environment. Spec Kit leaves remediation in the task list; OpenSpec requires reconciling change deltas with maintained specs. The examples here make the reviewer maintain the relationship map, scripts and host configuration. Deleting a stale mapping must therefore expose its former consumers rather than make them disappear from review. These are costs to measure in the trial, not evidence that one workflow is cheaper.
A hook runs at a boundary
A hook connects an event to an action. The host decides when the event occurs, whether the matcher selects it, what data the handler receives, and how the result affects the session. Those are separate contracts from the behaviour being reviewed.
Kiro hooks support commands and agent prompts. A command receives event context and can perform a deterministic check; an agent action adds an instruction for contextual work. For command actions, successful output becomes agent context. Its command-action contract sends nonzero-exit diagnostics to the agent and blocks at Pre Tool Use or Prompt Submit; its default timeout is 60 seconds. File and task events have different availability in the IDE and CLI. Use the trigger table for the surface being configured, rather than copying an older hook format into a newer client.
In Codex, command and MCP handlers are supported; the documented prompt and agent handler types are parsed but skipped. The adapter below therefore returns structured context from a command. It does not assume that a natural-language hook definition launches another reviewer.
Scroll horizontally to see all columns.
| Boundary | What it can establish | What remains |
|---|---|---|
| Pre-tool denial | The host prevented the selected invocation under the configured rule. | Other permitted paths may still exist; the rule’s coverage needs examination. |
| Post-tool notice | A selected tool invocation completed and the handler returned information. | The action already happened. The notice is a request for attention, not a conformance result. |
| Codex Stop continuation | The host asks the agent to continue after it tries to finish. | Continuation does not constitute a merge gate or prove that the requested review occurred. |
| Required repository check | The configured command ran under the repository’s check policy. | A green result is only as useful as its expectations, exercised boundary and recorded inputs. |
Codex documents concurrent execution of matching command hooks. Keep their state bounded and their effects independent; one handler cannot prevent another from starting. Deduplicating a notice reduces repeated context, but does not eliminate the cost of discovering whether anything changed.
Try the checks without hooks first
The examples are maintained here as copyable code rather than a separately versioned package. Save each marked block using its displayed filename in a disposable Git repository. Start with the checks that answer an actual review question. A structural linter, a relationship map and a run recorder have different jobs; adopting one does not require adopting the others.
Start in a disposable repository. If the checks earn a permanent place, the capability maintainer owns its map entries, and the author of a consumer or dependency change updates them in the same review. The reviewer checks those relationships against the affected paths; missing or stale scope remains unverified. Record the copied script revision or hash and assign its upgrades to the repository maintainer. A team that cannot justify that upkeep should remove the automation and retain the review it was meant to assist.
The examples target Python 3.10 or newer on macOS/Linux, using the standard library and POSIX process groups (subprocess documentation). They treat paths as repository-relative where they read Git state, use bounded scans and output, and report incomplete work. Validation here ran with Python 3.14.6 on macOS 26.6.2 on 2026-09-21. The Codex hook adapter has separate host-version compatibility notes because event payloads can change.
A small requirement check
Use stable IDs such as DEDUP-01 through DEDUP-05 and EXP-05 in the maintained contract. The checker only checks declarations, duplicates, empty definitions and explicit local references. It does not decide whether first-record retention or download authorisation is the right product rule.
Copy the marked fixture files too, then run python3 spec_lint.py specs/contact.md specs/authorisation.md. A successful run reports only structural success. A missing reference, duplicate ID or empty body is a failure that needs a human decision; adding text merely to satisfy the checker does not settle the rule.
Show the structural checker
File: spec_lint.py
#!/usr/bin/env python3
"""Check stable IDs and explicit local references in Markdown.
This is a structural check. It does not assess semantic correctness, scope,
or whether a decision has a sufficient basis.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Iterable
MAX_FILES = 64
MAX_BYTES = 1_048_576
MAX_FINDINGS = 200
MAX_DECLARATIONS = 2_048
IDENTIFIER = re.compile(r"^[A-Z][A-Z0-9]*(?:-[A-Z0-9]+)+$")
DECLARATION = re.compile(
r"^(?:[-*+]\s+)?(?P<identifier>[A-Z][A-Z0-9]*(?:-[A-Z0-9]+)+)\s*:\s*(?P<body>.*)$"
)
REFERENCE = re.compile(r"^\s{2,}(?:Covers|References|Refs)\s*:\s*(?P<body>.*)$")
@dataclass
class Finding:
severity: str
code: str
path: str
line: int
message: str
@dataclass
class Declaration:
identifier: str
path: str
line: int
body: str
references: list[tuple[str, int]] = field(default_factory=list)
@dataclass
class Result:
files: int = 0
declarations: int = 0
findings: list[Finding] = field(default_factory=list)
def add(self, severity: str, code: str, path: str, line: int, message: str) -> None:
if len(self.findings) < MAX_FINDINGS:
self.findings.append(Finding(severity, code, path, line, message))
elif not any(item.code == "too-many-findings" for item in self.findings):
self.findings.append(Finding(
"error", "too-many-findings", "", 0,
f"More than {MAX_FINDINGS} findings; details were bounded."
))
def exit_code(self) -> int:
return 1 if any(item.severity == "error" for item in self.findings) else 0
def read_lines(path: Path, result: Result) -> list[str]:
display = str(path)
if path.is_symlink():
result.add("error", "symlink-input", display, 0,
"Refusing a symlink; pass the real Markdown file.")
return []
if not path.exists():
result.add("error", "missing-input", display, 0, "Input does not exist.")
return []
if not path.is_file():
result.add("error", "not-a-file", display, 0,
"Only explicit Markdown files are scanned; directories and submodules are not traversed.")
return []
try:
size = path.stat().st_size
if size > MAX_BYTES:
result.add("error", "input-too-large", display, 0,
f"Input is {size} bytes; the limit is {MAX_BYTES}.")
return []
with path.open("rb") as handle:
raw = handle.read(MAX_BYTES + 1)
if len(raw) > MAX_BYTES:
result.add("error", "input-too-large", display, 0, "Input grew beyond its byte limit.")
return []
return raw.decode("utf-8").splitlines()
except (OSError, UnicodeError) as error:
result.add("error", "read-error", display, 0, str(error))
return []
def parse(path: Path, result: Result) -> list[Declaration]:
declarations: list[Declaration] = []
for number, line in enumerate(read_lines(path, result), 1):
match = DECLARATION.match(line)
if match:
if len(declarations) >= MAX_DECLARATIONS:
result.add("error", "too-many-declarations", str(path), number,
f"The limit is {MAX_DECLARATIONS} declarations.")
break
declarations.append(Declaration(
match.group("identifier"), str(path), number, match.group("body").strip()
))
continue
reference = REFERENCE.match(line)
if reference and not declarations:
result.add("error", "orphan-reference", str(path), number,
"A local reference must follow a declaration.")
continue
if reference and declarations:
body = reference.group("body").strip()
if not body:
result.add("error", "empty-reference", str(path), number,
"A local reference list cannot be empty.")
continue
parts = [part.strip() for part in body.split(",")]
if any(not IDENTIFIER.fullmatch(part) for part in parts):
result.add("error", "malformed-reference", str(path), number,
"References must be comma-separated stable IDs.")
else:
declarations[-1].references.extend((part, number) for part in parts)
continue
candidate = re.match(r"^(?:[-*+]\s+)?([A-Za-z][A-Za-z0-9_-]*)\s*:", line)
if candidate and candidate.group(1).upper().split("-", 1)[0] in {
"DEDUP", "EXP", "AC", "INV", "REQ", "AUTH"
}:
result.add("error", "malformed-declaration", str(path), number,
"Use an uppercase stable ID containing a hyphen, followed by a colon.")
return declarations
def scan(paths: Iterable[Path]) -> Result:
result = Result()
values = list(paths)
if not values:
result.add("error", "no-input", "", 0, "Pass at least one Markdown file.")
return result
if len(values) > MAX_FILES:
result.add("error", "too-many-files", "", 0, f"The limit is {MAX_FILES} files.")
return result
declarations: list[Declaration] = []
seen: set[Path] = set()
for supplied in values:
path = supplied.absolute()
if path in seen:
result.add("warning", "duplicate-input", str(supplied), 0,
"The same input was supplied more than once.")
continue
seen.add(path)
if path.suffix.lower() not in {".md", ".markdown"}:
result.add("error", "wrong-extension", str(supplied), 0,
"Expected a .md or .markdown file.")
continue
result.files += 1
declarations.extend(parse(path, result))
result.declarations = len(declarations)
if not declarations:
result.add("error", "no-declarations", "", 0,
"No stable-ID declarations were found in the supplied specification.")
homes: dict[str, list[Declaration]] = {}
for declaration in declarations:
homes.setdefault(declaration.identifier, []).append(declaration)
if not declaration.body:
result.add("error", "empty-definition", declaration.path, declaration.line,
f"{declaration.identifier} has an empty definition.")
for identifier, occurrences in homes.items():
if len(occurrences) > 1:
locations = ", ".join(f"{item.path}:{item.line}" for item in occurrences)
if len(locations) > 4_096:
locations = locations[:4_096] + "..."
for occurrence in occurrences:
result.add("error", "duplicate-id", occurrence.path, occurrence.line,
f"{identifier} has multiple homes: {locations}.")
for declaration in declarations:
for target, line in declaration.references:
if target not in homes:
result.add("error", "dangling-reference", declaration.path, line,
f"{declaration.identifier} refers to missing {target}.")
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("paths", nargs="+", type=Path)
parser.add_argument("--json", action="store_true")
args = parser.parse_args(argv)
result = scan(args.paths)
if args.json:
print(json.dumps(asdict(result), indent=2, ensure_ascii=False))
else:
print(f"Checked {result.files} file(s) and {result.declarations} declaration(s).")
for finding in result.findings:
location = f"{finding.path}:{finding.line}" if finding.path else "input"
print(f"{location}: {finding.severity} {finding.code}: {finding.message}")
if not result.findings:
print("No structural findings. Semantic correctness was not assessed.")
return result.exit_code()
if __name__ == "__main__":
sys.exit(main())
File: specs/contact.md
# Contact deduplication
DEDUP-01: Equal non-empty string contact_id values identify duplicates using exact case-sensitive comparison.
DEDUP-02: Retain the first complete record for each ID in original order.
DEDUP-03: Do not mutate input on success or rejection.
DEDUP-04: Reject the whole input when contact_id is missing, empty, or non-string.
DEDUP-05: Empty input succeeds with an empty result.
Covers: DEDUP-01, DEDUP-02, DEDUP-03, DEDUP-04, DEDUP-05
File: specs/authorisation.md
# Invoice download authorisation
EXP-05: Each new download request requires current authorisation for the account,
including requests through alternate direct-download entry points.
Covers: EXP-05
Run a small repair fixture
The following fixture gives the examples a real boundary to exercise. The stipulated fixture contract is the five DEDUP-* rules above. The authorisation fixture uses already-resolved account/object inputs; it does not model URL parsing or object ownership. Its tests include a direct-download entry point as well as the ordinary API path: a test that only calls the API can pass while the alternate path still bypasses revocation.
Save the marked blocks at their displayed paths, then run:
python3 -B -m unittest discover -s fixtures -p 'test_*.py' -v
Change deduplicate so that a later duplicate overwrites the retained record; the first-record test fails. Restore that implementation, then make direct_download return the object without checking permission. Run only the API permission test: it still passes despite the bypass. The full suite detects the broken alternate path. Restore the authorisation check before recording passing evidence.
Contact and download fixtures with their executable checks
File: fixtures/__init__.py
"""Disposable examples for the inline SDD checks."""
File: fixtures/contact_dedup.py
"""A deliberately small implementation of DEDUP-01 through DEDUP-05."""
from __future__ import annotations
def deduplicate(records: list[dict[str, object]]) -> list[dict[str, object]]:
"""Retain the first record for each exact, non-empty contact_id."""
retained: list[dict[str, object]] = []
seen: set[str] = set()
for position, record in enumerate(records):
if not isinstance(record, dict):
raise ValueError(f"record {position} is not an object")
identifier = record.get("contact_id")
if not isinstance(identifier, str) or not identifier:
raise ValueError(f"record {position} has an invalid contact_id")
if identifier in seen:
continue
seen.add(identifier)
retained.append(record.copy())
return retained
File: fixtures/auth_boundary.py
"""Two download entry points sharing one current-authorisation check."""
from __future__ import annotations
def _download(key: str, account: str, principal: str,
permissions: dict[str, set[str]], objects: dict[str, bytes]) -> bytes:
if account not in permissions.get(principal, set()):
raise PermissionError("current account permission required")
return objects[key]
def api_download(account: str, principal: str, key: str,
permissions: dict[str, set[str]], objects: dict[str, bytes]) -> bytes:
return _download(key, account, principal, permissions, objects)
def direct_download(account: str, principal: str, key: str,
permissions: dict[str, set[str]], objects: dict[str, bytes]) -> bytes:
return _download(key, account, principal, permissions, objects)
File: fixtures/test_contracts.py
from __future__ import annotations
import unittest
from contact_dedup import deduplicate
from auth_boundary import api_download, direct_download
class ContactContractTests(unittest.TestCase):
def test_first_record_and_relative_order_are_preserved(self) -> None:
records = [
{"contact_id": "a", "name": "Ada"},
{"contact_id": "b", "name": "Bo"},
{"contact_id": "a", "name": "Amal"},
]
original = [record.copy() for record in records]
self.assertEqual(deduplicate(records), [records[0], records[1]])
self.assertEqual(records, original)
def test_case_sensitive_identity_and_invalid_input(self) -> None:
self.assertEqual(
deduplicate([{"contact_id": "a"}, {"contact_id": "A"}]),
[{"contact_id": "a"}, {"contact_id": "A"}],
)
self.assertEqual(deduplicate([]), [])
for invalid in ({}, {"contact_id": ""}, {"contact_id": None}, {"contact_id": 7}):
records = [{"contact_id": "a"}, invalid]
original = [record.copy() for record in records]
with self.subTest(invalid=invalid):
with self.assertRaises(ValueError):
deduplicate(records)
self.assertEqual(records, original)
class DownloadContractTests(unittest.TestCase):
def setUp(self) -> None:
self.permissions = {"admin": {"acct-1"}}
self.objects = {"export-1": b"invoice"}
def test_api_download_requires_current_permission(self) -> None:
self.assertEqual(api_download("acct-1", "admin", "export-1", self.permissions, self.objects), b"invoice")
self.permissions["admin"].remove("acct-1")
with self.assertRaises(PermissionError):
api_download("acct-1", "admin", "export-1", self.permissions, self.objects)
def test_direct_download_also_requires_current_permission(self) -> None:
self.permissions["admin"].remove("acct-1")
with self.assertRaises(PermissionError):
direct_download("acct-1", "admin", "export-1", self.permissions, self.objects)
if __name__ == "__main__":
unittest.main()
Record evidence without treating it as approval
Use an external record directory so a result can be compared after another agent or merge changes the workspace:
SDD_RECORDS=$(mktemp -d)
python3 check_run.py run --record-dir "$SDD_RECORDS" --cwd . --timeout 30 -- \
python3 -B -m unittest discover -s fixtures -p 'test_*.py'
python3 check_run.py compare "$SDD_RECORDS/"*.json
compare answers whether the complete before, after and current workspace fingerprints agree. It reports matches_recorded_inputs separately from recorded_command_passed: a failed command can have matching inputs, and a passing command can have stale evidence. A command that changes the workspace reports workspace_changed_during_run. Inspect both the result and the input comparison before using the record in review.
run retains the exact argument vector (up to 64 KiB encoded), up to 64 KiB each of stdout and stderr, return code, timeout state and workspace fingerprints. The snapshot includes ordinary and ignored files and directory entries under the selected working directory, including the root directory. It hashes relative paths, entry types, file contents and POSIX permission bits, excluding .git and the external record directory. It neither follows symlinks nor enters Git submodules. Scans are limited to 4,096 files, 4,096 directories including the root, 32 MiB and ten seconds per snapshot; incomplete scans remain visible in the record even when the command succeeds.
The before/after observations are not an atomic filesystem snapshot. Stop other writers while collecting evidence; an edit reverted during the command can escape comparison. Timestamps, ownership, ACLs and extended attributes are excluded. The recorder identifies its Python runtime, but does not fingerprint environment variables, installed runtimes, external services or files outside the workspace. Matching inputs therefore covers only the recorded scope. The record is unsigned, so the reviewer must trust its origin. Retain the external record directory for as long as the review needs it.
The recorder now writes record format version 2. Version 1 omitted directory state; compare rejects those records and asks for a fresh run. The historical results in Part 6 remain observations from the earlier format, not inputs to the current comparator.
Show the bounded execution recorder
File: check_run.py
#!/usr/bin/env python3
"""Record a bounded command and detect whether its workspace evidence is stale."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import selectors
import stat
import math
import signal
import subprocess
import sys
import tempfile
import time
import uuid
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
MAX_RECORD_BYTES = 2 * 1024 * 1024
RECORD_SCHEMA_VERSION = 2
MAX_FILES = 4_096
MAX_DIRECTORIES = 4_096
MAX_SNAPSHOT_BYTES = 32 * 1024 * 1024
MAX_FILE_HASH_BYTES = 1 * 1024 * 1024
DEFAULT_TIMEOUT = 60.0
DEFAULT_OUTPUT = 64 * 1024
SNAPSHOT_TIMEOUT = 10.0
def error(message: str) -> RuntimeError:
return RuntimeError(message)
def hash_file(path: Path) -> tuple[str, int, bool, str | None]:
"""Hash a bounded prefix without following symlinks."""
digest = hashlib.sha256()
try:
if path.is_symlink():
target = os.readlink(path).encode("utf-8", "surrogateescape")
digest.update(b"symlink\0")
digest.update(target)
return digest.hexdigest(), len(target), False, "symlink-target-not-followed"
info = path.stat()
if not stat.S_ISREG(info.st_mode):
return "", 0, False, "not-a-regular-file"
size = info.st_size
digest.update(str(stat.S_IMODE(info.st_mode)).encode())
digest.update(b"\0")
with path.open("rb") as handle:
data = handle.read(MAX_FILE_HASH_BYTES + 1)
complete = size <= MAX_FILE_HASH_BYTES and len(data) <= MAX_FILE_HASH_BYTES
digest.update(str(size).encode())
digest.update(b"\0")
digest.update(data[:MAX_FILE_HASH_BYTES])
return digest.hexdigest(), size, complete, "large-file" if not complete else None
except OSError as exc:
digest.update(f"error:{exc}".encode())
return digest.hexdigest(), 0, False, str(exc)
def scan_workspace(root: Path) -> dict[str, Any]:
root = root.resolve()
digest = hashlib.sha256()
complete = True
file_count = directory_count = byte_count = hashed_bytes = 0
warnings: list[str] = []
def warn(message):
if len(warnings) < 64:
warnings.append(message[:512])
elif len(warnings) == 64:
warnings.append("additional warnings omitted")
records = []
if not root.is_dir():
raise error(f"workspace is not a directory: {root}")
records.append((".", "directory", stat.S_IMODE(root.stat().st_mode)))
directory_count = 1
deadline = time.monotonic() + SNAPSHOT_TIMEOUT
pending = [root]
while pending:
if time.monotonic() > deadline:
raise error(f"workspace snapshot exceeded {SNAPSHOT_TIMEOUT} seconds")
base = pending.pop()
with os.scandir(base) as entries:
for entry in entries:
if time.monotonic() > deadline:
raise error(f"workspace snapshot exceeded {SNAPSHOT_TIMEOUT} seconds")
if entry.name == ".git":
continue
path = Path(entry.path)
relative = path.relative_to(root).as_posix()
if entry.is_dir(follow_symlinks=False):
directory_count += 1
if directory_count > MAX_DIRECTORIES:
raise error(f"workspace has more than {MAX_DIRECTORIES} directories")
mode = stat.S_IMODE(entry.stat(follow_symlinks=False).st_mode)
records.append((relative, "directory", mode))
if (path / ".git").exists():
complete = False
warn(f"{relative}: nested repository excluded")
else:
pending.append(path)
continue
file_count += 1
if file_count > MAX_FILES:
raise error(f"workspace has more than {MAX_FILES} files")
if hashed_bytes >= MAX_SNAPSHOT_BYTES:
raise error("snapshot byte limit reached")
value, size, file_complete, reason = hash_file(path)
byte_count += size
hashed_bytes += min(size, MAX_FILE_HASH_BYTES)
if hashed_bytes > MAX_SNAPSHOT_BYTES:
raise error("snapshot byte limit reached")
records.append((relative, "file", value, size))
if not file_complete:
complete = False
warn(f"{relative}: {reason}")
for record in sorted(records):
digest.update(json.dumps(record, ensure_ascii=True).encode())
return {
"algorithm": "sha256", "value": digest.hexdigest(), "complete": complete,
"files": file_count, "directories": directory_count, "bytes": byte_count,
"hashed_files": file_count, "ignored_files_included": True,
"git_metadata_excluded": True, "warnings": warnings,
}
def snapshot(root: Path, excluded: Path) -> dict[str, Any]:
try:
return scan_workspace(root)
except (OSError, RuntimeError) as failure:
return {"algorithm": "sha256", "value": "", "complete": False,
"warnings": [str(failure)[:1000]]}
def bounded_process(command: list[str], cwd: Path, timeout: float, output_limit: int) -> dict[str, Any]:
if not command:
raise error("a command is required after '--'")
process = subprocess.Popen(command, cwd=cwd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, start_new_session=True)
captured = {"stdout": bytearray(), "stderr": bytearray()}
truncated = {"stdout": False, "stderr": False}
deadline = time.monotonic() + timeout
timed_out = False
try:
with selectors.DefaultSelector() as selector:
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
while selector.get_map() or process.poll() is None:
remaining = deadline - time.monotonic()
if remaining <= 0:
timed_out = True
break
for key, _ in selector.select(min(remaining, 0.1)):
data = os.read(key.fileobj.fileno(), 8192)
if not data:
selector.unregister(key.fileobj)
continue
name = key.data
room = output_limit - len(captured[name])
captured[name].extend(data[:max(0, room)])
if len(data) > room:
truncated[name] = True
finally:
# A recorded command cannot leave descendants running after the run.
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
process.wait(timeout=2)
process.stdout.close()
process.stderr.close()
return {
"returncode": process.returncode, "timed_out": timed_out,
"stdout": captured["stdout"].decode("utf-8", "replace"),
"stderr": captured["stderr"].decode("utf-8", "replace"),
"stdout_truncated": truncated["stdout"], "stderr_truncated": truncated["stderr"],
}
def record_path(directory: Path) -> Path:
if directory.exists() and directory.is_symlink():
raise error("record directory cannot be a symlink")
directory.mkdir(parents=True, exist_ok=True)
if not directory.is_dir():
raise error("record path is not a directory")
return directory / f"{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}-{uuid.uuid4().hex[:10]}.json"
def write_record(path: Path, record: dict[str, Any]) -> None:
encoded = json.dumps(record, indent=2, ensure_ascii=False).encode("utf-8")
if len(encoded) > MAX_RECORD_BYTES:
raise error(f"record exceeds {MAX_RECORD_BYTES} bytes")
with tempfile.NamedTemporaryFile("wb", dir=path.parent, prefix=".sdd-record-", delete=False) as handle:
temporary = Path(handle.name)
handle.write(encoded)
temporary.replace(path)
def run_command(args: argparse.Namespace) -> int:
cwd = args.cwd.resolve()
record_dir = args.record_dir.resolve()
if record_dir == cwd or cwd in record_dir.parents:
raise error("record directory must be outside the workspace")
before = snapshot(cwd, record_dir)
result = bounded_process(args.command, cwd, args.timeout, args.max_output)
after = snapshot(cwd, record_dir)
path = record_path(record_dir)
record = {
"schema_version": RECORD_SCHEMA_VERSION,
"runtime": {"python": sys.version, "executable": sys.executable, "platform": sys.platform},
"recorded_at": datetime.now(timezone.utc).isoformat(),
"command": args.command,
"cwd": str(cwd),
"record_dir": str(record_dir),
"timeout_seconds": args.timeout,
"max_output_bytes": args.max_output,
"result": result,
"recorded_command_passed": result["returncode"] == 0 and not result["timed_out"],
"workspace_changed_during_run": before["value"] != after["value"],
"before": before,
"after": after,
"limitations": [
"This is an unsigned local record; it does not authenticate approval.",
"Ignored files are included, while .git metadata and submodule contents are excluded.",
"Timestamps, ownership, ACLs, and extended attributes are not fingerprinted.",
"The recorder's Python runtime is identified but its contents are not fingerprinted.",
"The process environment, external services, and filesystem changes outside cwd are not recorded.",
],
}
write_record(path, record)
print(json.dumps({
"record": str(path),
"recorded_command_passed": record["recorded_command_passed"],
"before_complete": before["complete"],
"after_complete": after["complete"],
}, indent=2))
if not record["recorded_command_passed"]:
return 1
return 0 if before["complete"] and after["complete"] else 2
def compare_record(args: argparse.Namespace) -> int:
path = args.record.absolute()
if path.is_symlink() or not path.is_file() or path.stat().st_size > MAX_RECORD_BYTES:
raise error("record must be a regular file within the record-size limit")
try:
with path.open("rb") as handle:
raw = handle.read(MAX_RECORD_BYTES + 1)
if len(raw) > MAX_RECORD_BYTES:
raise error("record exceeds its byte limit")
record = json.loads(raw)
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise error(f"cannot read record: {exc}") from exc
if isinstance(record, dict) and record.get("schema_version") == 1:
raise error("record format version 1 omitted directory state; run the command again to create a version-2 record")
if (not isinstance(record, dict) or type(record.get("schema_version")) is not int
or record["schema_version"] != RECORD_SCHEMA_VERSION):
raise error("unsupported record schema; run the command again to create a version-2 record")
if not all(isinstance(record.get(key), str) for key in ("cwd", "record_dir")):
raise error("record requires cwd and record_dir strings")
for key in ("before", "after"):
value = record.get(key)
if (not isinstance(value, dict) or not isinstance(value.get("complete"), bool)
or not isinstance(value.get("value"), str)):
raise error(f"invalid {key} fingerprint")
if not isinstance(record.get("recorded_command_passed"), bool):
raise error("record requires a command-success boolean")
cwd = Path(record["cwd"]).resolve()
record_dir = Path(record["record_dir"]).resolve()
current = snapshot(cwd, record_dir)
recorded_after = record.get("after", {})
recorded_before = record.get("before", {})
matches = bool(
recorded_before.get("complete") and recorded_after.get("complete") and current["complete"]
and recorded_before.get("value") == recorded_after.get("value") == current["value"]
)
payload = {
"record": str(path),
"matches_recorded_inputs": matches,
"recorded_command_passed": bool(record.get("recorded_command_passed")),
"workspace_changed_during_run": bool(record.get("workspace_changed_during_run")),
"current": current,
"recorded_after": recorded_after,
"command": record.get("command"),
"limitations": record.get("limitations", []),
}
print(json.dumps(payload, indent=2, ensure_ascii=False))
return 0 if matches else 1
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
commands = parser.add_subparsers(dest="operation", required=True)
run = commands.add_parser("run")
run.add_argument("--record-dir", required=True, type=Path)
run.add_argument("--cwd", type=Path, default=Path("."))
run.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT)
run.add_argument("--max-output", type=int, default=DEFAULT_OUTPUT)
run.add_argument("command", nargs=argparse.REMAINDER)
compare = commands.add_parser("compare")
compare.add_argument("record", type=Path)
args = parser.parse_args(argv)
try:
if args.operation == "run":
if args.command and args.command[0] == "--":
args.command = args.command[1:]
if not math.isfinite(args.timeout) or args.timeout <= 0 or args.timeout > 600:
raise error("timeout must be greater than zero and no more than 600 seconds")
if args.max_output <= 0 or args.max_output > 65_536:
raise error("max-output must be between 1 and 65536 bytes")
if len(json.dumps(args.command, ensure_ascii=True).encode()) > 65_536:
raise error("encoded command arguments exceed 64 KiB")
return run_command(args)
return compare_record(args)
except (OSError, RuntimeError, ValueError, subprocess.SubprocessError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())
Select affected specifications
The selector takes a Git baseline explicitly. Save the relationship map below as spec-map.json near the contract; it records dependencies without copying the rules. In a disposable repository containing the extracted blocks, establish the baseline before editing:
git init
git add .
git commit -m 'fixture baseline'
git tag review-base
Run python3 spec_notice.py --root . --baseline review-base --map spec-map.json --json. The result lists changed paths, affected specifications and unmatched paths. It compares the current map with the map at review-base, includes staged, unstaged and untracked paths, reports renames, and marks a missing baseline map as incomplete. Ignored paths and Git submodules are warnings because this small example cannot infer their content. The selector has a twelve-second overall deadline, a 4 MiB cap per Git output stream and a 1 MiB JSON report cap. Exceeding a limit produces an incomplete/error result. The workspace_fingerprint changes when bounded mapped or changed files change; workspace_fingerprint_complete tells a hook whether it is safe to deduplicate a notice.
Every target in the current map must exist. A missing specification, implementation path or consumer produces a warning naming its role and contract, sets scope_complete and complete to false, and exits with status 1. Known absence can still be fingerprinted completely; that does not make the relationship valid. Baseline-only relationships remain historical review candidates, so removing a mapping and its former files does not itself make the current scope incomplete. Even a complete: true notice does not prove that the map includes every relevant consumer.
File: spec-map.json
{
"version": 1,
"relationships": [
{
"spec": "specs/contact.md",
"paths": ["fixtures/contact_dedup.py"],
"consumers": ["fixtures/test_contracts.py"]
},
{
"spec": "specs/authorisation.md",
"paths": ["fixtures/auth_boundary.py"],
"consumers": ["fixtures/test_contracts.py"]
}
]
}
Show the affected-specification selector
File: spec_notice.py
#!/usr/bin/env python3
"""Select review candidates from Git changes and a relationship map.
Selection is bookkeeping. It does not inspect implementation correctness or
authenticate an approval. A complete result means the requested inputs were
read within their bounds, not that every repository dependency was understood.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import selectors
import signal
import stat
import subprocess
import sys
import time
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import Any
MAX_MAP_BYTES = 512 * 1024
MAX_RELATIONSHIPS = 512
MAX_CHANGED = 2_000
MAX_FINGERPRINT_FILES = 256
MAX_FINGERPRINT_BYTES = 16 * 1024 * 1024
MAX_FILE_HASH_BYTES = 2 * 1024 * 1024
MAX_GIT_OUTPUT = 4 * 1024 * 1024
GIT_TIMEOUT = 10.0
SCAN_TIMEOUT = 12.0
MAX_REPORT_BYTES = 1024 * 1024
def fail(message: str) -> RuntimeError:
return RuntimeError(message)
def safe_path(value: Any) -> str:
if not isinstance(value, str) or not value.strip():
raise fail("relationship paths must be non-empty strings")
normal = value.replace("\\", "/")
path = PurePosixPath(normal)
if (path.is_absolute() or PureWindowsPath(value).is_absolute()
or any(part in {"", ".", ".."} for part in path.parts)):
raise fail(f"path must be repository-relative without traversal: {value!r}")
return path.as_posix()
def parse_mapping(data: Any) -> dict[str, dict[str, list[str]]]:
if not isinstance(data, dict) or type(data.get("version")) is not int or data.get("version") != 1:
raise fail("mapping must be an object with version 1")
if set(data) != {"version", "relationships"}:
raise fail("mapping requires only version and relationships")
relationships = data.get("relationships")
if not isinstance(relationships, list) or len(relationships) > MAX_RELATIONSHIPS:
raise fail(f"relationships must be a list of at most {MAX_RELATIONSHIPS} entries")
output: dict[str, dict[str, list[str]]] = {}
for item in relationships:
if not isinstance(item, dict):
raise fail("each relationship must be an object")
if set(item) != {"spec", "paths", "consumers"}:
raise fail("relationship requires spec, paths and consumers")
spec = safe_path(item.get("spec"))
if spec in output:
raise fail(f"duplicate relationship for {spec}")
values: dict[str, list[str]] = {}
for label in ("paths", "consumers"):
entries = item.get(label, [])
if not isinstance(entries, list) or len(entries) > MAX_FINGERPRINT_FILES:
raise fail(f"{spec}: {label} must be a list")
values[label] = []
for entry in entries:
normalized = safe_path(entry)
if normalized not in values[label]:
values[label].append(normalized)
output[spec] = values
return output
def read_json(path: Path) -> dict[str, dict[str, list[str]]]:
if path.is_symlink():
raise fail("refusing a symlink mapping file")
if not path.is_file() or path.stat().st_size > MAX_MAP_BYTES:
raise fail(f"mapping must be a regular file no larger than {MAX_MAP_BYTES} bytes")
try:
with path.open("rb") as handle:
raw = handle.read(MAX_MAP_BYTES + 1)
if len(raw) > MAX_MAP_BYTES:
raise fail("map exceeds its byte limit")
return parse_mapping(json.loads(raw))
except (OSError, UnicodeError, json.JSONDecodeError) as error:
raise fail(f"cannot read mapping: {error}") from error
def git(root: Path, *arguments: str) -> str:
"""Run Git with bounded stdout/stderr; path listings can be large."""
try:
process = subprocess.Popen(
["git", *arguments], cwd=root, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
start_new_session=True,
)
except OSError as error:
raise fail(f"git {' '.join(arguments)} failed: {error}") from error
assert process.stdout is not None and process.stderr is not None
selector = selectors.DefaultSelector()
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
captured = {"stdout": bytearray(), "stderr": bytearray()}
deadline = time.monotonic() + GIT_TIMEOUT
try:
while selector.get_map():
remaining = deadline - time.monotonic()
if remaining <= 0:
process.kill()
process.wait()
raise fail(f"git {' '.join(arguments)} exceeded {GIT_TIMEOUT} seconds")
for key, _ in selector.select(min(remaining, 0.25)):
data = os.read(key.fileobj.fileno(), 8192)
if not data:
selector.unregister(key.fileobj)
key.fileobj.close()
continue
captured[key.data].extend(data)
if len(captured[key.data]) > MAX_GIT_OUTPUT:
process.kill()
process.wait()
raise fail(f"git {' '.join(arguments)} exceeded the {MAX_GIT_OUTPUT}-byte output limit")
process.wait(timeout=max(0.001, deadline - time.monotonic()))
finally:
selector.close()
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
process.wait(timeout=2)
process.stdout.close()
process.stderr.close()
stdout = bytes(captured["stdout"]).decode("utf-8", "replace")
stderr = bytes(captured["stderr"]).decode("utf-8", "replace")
if process.returncode:
detail = stderr.strip() or stdout.strip() or f"exit {process.returncode}"
raise fail(f"git {' '.join(arguments)} failed: {detail}")
return stdout
def parse_diff(raw: str) -> list[dict[str, str]]:
values = raw.split("\0")
if values and values[-1] == "":
values.pop()
output: list[dict[str, str]] = []
index = 0
while index < len(values):
status = values[index]
index += 1
if not status:
continue
if status[:1] in {"R", "C"} and index + 1 < len(values):
old, new = values[index], values[index + 1]
index += 2
output.append({"status": status, "path": new, "old_path": old})
elif index < len(values):
output.append({"status": status, "path": values[index]})
index += 1
return output
def changed_paths(root: Path, baseline: str) -> list[dict[str, str]]:
values = parse_diff(git(root, "diff", "--name-status", "-M", "-z", baseline, "--"))
seen = {(item["path"], item.get("old_path")) for item in values}
status_values = git(root, "status", "--porcelain=v1", "-z", "--untracked-files=all").split("\0")
index = 0
while index < len(status_values):
entry = status_values[index]
index += 1
if not entry:
continue
if len(entry) < 4:
raise fail(f"unparseable Git status entry: {entry!r}")
flags, path = entry[:2], entry[3:]
item = {"status": flags, "path": path}
if flags[:1] in {"R", "C"} and index < len(status_values):
# Porcelain -z gives the destination first and the original second.
item["old_path"] = status_values[index]
index += 1
key = (item["path"], item.get("old_path"))
if key not in seen:
values.append(item)
seen.add(key)
if len(values) > MAX_CHANGED:
raise fail(f"workspace has more than {MAX_CHANGED} changed paths")
return values
def baseline_mapping(root: Path, baseline: str, mapping_path: str) -> dict[str, dict[str, list[str]]] | None:
if not git(root, "ls-tree", "--name-only", "-z", baseline, "--", mapping_path):
return None
raw = git(root, "show", f"{baseline}:{mapping_path}")
if len(raw.encode("utf-8")) > MAX_MAP_BYTES:
raise fail("baseline map exceeds the map-size limit")
return parse_mapping(json.loads(raw))
def map_delta(current: dict[str, dict[str, list[str]]],
previous: dict[str, dict[str, list[str]]] | None) -> dict[str, list[str]]:
if previous is None:
return {"added": sorted(current), "removed": [], "changed": []}
added = sorted(set(current) - set(previous))
removed = sorted(set(previous) - set(current))
changed = sorted(name for name in set(current) & set(previous) if current[name] != previous[name])
return {"added": added, "removed": removed, "changed": changed}
def missing_current_targets(root: Path, mapping: dict[str, dict[str, list[str]]]) -> list[str]:
warnings = []
for spec, relation in sorted(mapping.items()):
for role, paths in (("specification", [spec]), ("implementation", relation["paths"]),
("consumer", relation["consumers"])):
for relative in sorted(paths):
if not (root / relative).exists():
warnings.append(f"{spec}: missing current {role} target {relative!r}")
return warnings
def fingerprint(root: Path, changed: list[dict[str, str]], mapping: dict[str, dict[str, list[str]]]) -> tuple[str, bool, list[str], list[str]]:
paths = {item["path"] for item in changed}
for item in changed:
if item.get("old_path"):
paths.add(item["old_path"])
for spec, relation in mapping.items():
paths.add(spec)
paths.update(relation["paths"])
paths.update(relation["consumers"])
ordered = sorted(paths)
warnings: list[str] = []
complete = True
if len(ordered) > MAX_FINGERPRINT_FILES:
ordered = ordered[:MAX_FINGERPRINT_FILES]
complete = False
warnings.append(f"fingerprint limited to {MAX_FINGERPRINT_FILES} paths")
digest = hashlib.sha256()
used = 0
for relative in ordered:
path = root / relative
digest.update(relative.encode("utf-8", "surrogateescape"))
digest.update(b"\0")
try:
if path.is_symlink() or any(parent.is_symlink() for parent in path.parents if root in parent.parents):
data = b"symlink-input-not-followed"
file_complete = False
elif path.is_file():
digest.update(str(stat.S_IMODE(path.stat().st_mode)).encode())
with path.open("rb") as handle:
data = handle.read(MAX_FILE_HASH_BYTES + 1)
file_complete = len(data) <= MAX_FILE_HASH_BYTES and path.stat().st_size <= MAX_FILE_HASH_BYTES
else:
data = f"missing:{relative}".encode()
file_complete = not path.exists()
except OSError as error:
data = f"error:{relative}".encode()
file_complete = False
warnings.append(f"{relative}: {error}")
digest.update(hashlib.sha256(data[:MAX_FILE_HASH_BYTES]).digest())
used += len(relative) + len(data)
if not file_complete:
complete = False
warnings.append(f"{relative}: content hash was bounded")
if used > MAX_FINGERPRINT_BYTES:
complete = False
warnings.append(f"fingerprint input exceeded {MAX_FINGERPRINT_BYTES} bytes")
break
return digest.hexdigest(), complete, ordered, warnings
def scan_expired(_signal, _frame):
raise TimeoutError("notice scan exceeded 12 seconds")
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=Path("."))
parser.add_argument("--baseline", required=True)
parser.add_argument("--mapping", "--map", dest="mapping", required=True, type=Path)
parser.add_argument("--json", action="store_true")
args = parser.parse_args(argv)
signal.signal(signal.SIGALRM, scan_expired)
signal.setitimer(signal.ITIMER_REAL, SCAN_TIMEOUT)
try:
root = args.root.resolve()
baseline = git(root, "rev-parse", "--verify", "--end-of-options", args.baseline + "^{commit}").strip()
mapping_path = args.mapping if not args.mapping.is_absolute() else args.mapping.resolve().relative_to(root)
mapping_name = safe_path(mapping_path.as_posix())
current_path = root / mapping_name
if any(parent.is_symlink() for parent in current_path.parents if root in parent.parents):
raise fail("mapping path traverses a symlink")
changed = changed_paths(root, baseline)
previous = baseline_mapping(root, baseline, mapping_name)
current_error = None
if not current_path.exists() and previous is not None:
# A deleted map is still a useful signal. Keep the baseline
# relationships visible so their former consumers are reviewable.
current = {}
current_error = f"current relationship map {mapping_name!r} is missing"
else:
current = read_json(current_path)
delta = map_delta(current, previous)
changed_names = {item["path"] for item in changed}
changed_names.update(item["old_path"] for item in changed if item.get("old_path"))
affected: list[dict[str, Any]] = []
mapped: set[str] = {mapping_name}
fingerprint_mapping = {}
all_specs = set(current) | set(previous or {})
for spec in sorted(all_specs):
relation = current.get(spec) or (previous or {})[spec]
old_relation = (previous or {}).get(spec)
related = {spec, *relation["paths"], *relation["consumers"]}
if old_relation:
related.update(old_relation["paths"])
related.update(old_relation["consumers"])
mapped.update(related)
fingerprint_mapping[spec] = {"paths": sorted(related - {spec}), "consumers": []}
touched = sorted(related & changed_names)
reasons = (["changed-path"] if touched else [])
if spec in delta["added"]:
reasons.append("mapping-added")
if spec in delta["changed"]:
reasons.append("mapping-changed")
if spec in delta["removed"]:
reasons.append("mapping-removed")
if reasons:
item = {
"spec": spec,
"reasons": sorted(set(reasons)),
"paths": relation["paths"],
"consumers": relation["consumers"],
"changed_paths": touched,
}
if old_relation and old_relation != relation:
item["previous_paths"] = old_relation["paths"]
item["previous_consumers"] = old_relation["consumers"]
affected.append(item)
unmatched = sorted(changed_names - mapped)
warnings = missing_current_targets(root, current)
coverage_complete = not warnings
if previous is None:
warnings.append("baseline relationship map is missing; scope is incomplete")
coverage_complete = False
if current_error:
warnings.append(current_error)
coverage_complete = False
if unmatched:
warnings.append(f"{len(unmatched)} changed path(s) have no relationship entry")
coverage_complete = False
ignored = set(git(root, "ls-files", "--others", "--ignored", "--exclude-standard", "-z").split("\0"))
for path in sorted(mapped):
if path in ignored:
warnings.append(f"{path}: ignored untracked changes are invisible to Git status")
coverage_complete = False
if (root / path).is_symlink():
warnings.append(f"{path}: symlink content is not followed")
coverage_complete = False
if (root / path).is_dir() and (root / path / ".git").exists():
warnings.append(f"{path}: Git submodule contents are outside this scan")
coverage_complete = False
value, hash_complete, hash_paths, hash_warnings = fingerprint(root, changed, fingerprint_mapping)
warnings.extend(hash_warnings)
payload = {
"schema_version": 1,
"complete": previous is not None and hash_complete and coverage_complete,
"scan_complete": hash_complete,
"scope_complete": previous is not None and hash_complete and coverage_complete,
"baseline": args.baseline,
"baseline_commit": baseline,
"mapping": mapping_name,
"changed_paths": changed,
"changed": sorted(changed_names),
"affected": affected,
"unmatched": unmatched,
"mapping_delta": delta,
"workspace_fingerprint": value,
"workspace_fingerprint_complete": hash_complete,
"fingerprint_paths": hash_paths,
"warnings": [{"severity": "warning", "message": warning} for warning in warnings],
}
code = 0 if payload["complete"] else 1
except (OSError, RuntimeError, ValueError, subprocess.SubprocessError) as error:
payload = {
"schema_version": 1,
"complete": False,
"scan_complete": False,
"scope_complete": False,
"changed_paths": [],
"affected": [],
"unmatched": [],
"warnings": [{"severity": "error", "code": "input-error", "message": str(error)}],
}
code = 2
finally:
signal.setitimer(signal.ITIMER_REAL, 0)
encoded = json.dumps(payload, indent=2, ensure_ascii=True)
if len(encoded.encode()) > MAX_REPORT_BYTES:
payload = {"complete": False, "changed_paths": [], "affected": [], "unmatched": [],
"warnings": [{"severity": "error", "message": "notice report exceeded 1 MiB"}]}
encoded = json.dumps(payload)
code = 2
if args.json:
print(encoded)
else:
print(f"Scope complete: {payload.get('scope_complete', False)}")
for item in payload.get("affected", []):
print(f"{item['spec']}: {', '.join(item['reasons'])}")
for warning in payload.get("warnings", []):
print(f"{warning['severity']}: {warning.get('message', '')}")
return code
if __name__ == "__main__":
sys.exit(main())
Use the same checks in an existing CI job
After copying the relevant blocks into the repository, an existing CI job can run the ordinary commands. Keep the record directory outside the checkout and retain it through the job’s usual artifact mechanism. Configure the job as a required check if repository policy calls for it; printing a notice creates no merge requirement.
set -eu
SDD_RECORDS=$(mktemp -d)
python3 spec_lint.py specs/contact.md specs/authorisation.md
python3 check_run.py run --record-dir "$SDD_RECORDS" --cwd . -- \
python3 -B -m unittest discover -s fixtures -p 'test_*.py'
for record in "$SDD_RECORDS"/*.json; do
python3 check_run.py compare "$record"
done
run failing stops this job even when its inputs match. A later input change makes compare fail even when the earlier command passed. The job’s artifact retention and cleanup policy owns the external record directory.
Attach a notice to Codex
Compatibility: native delivery was exercised with Codex CLI 0.155.1, model gpt-6-astra at xhigh, on 21 September 2026. Event and trust behaviour follows the Codex hooks documentation; the observations and their limits are recorded below.
The adapter accepts JSON on standard input, calls spec_notice.py, and returns Codex’s event-specific output. It runs from the repository root even when the session starts in a subdirectory. It never interprets a transcript or a model’s final sentence as evidence that review finished.
Save codex_notice.py alongside the other scripts. Choose a fixed baseline for the review and an explicit state file outside the repository. The example below uses a local review-base tag; create it at the chosen commit before starting work. Moving that tag changes the comparison.
The state file keeps at most 128 session/repository entries. A database transaction serialises concurrent notifications. Duplicate suppression records that output was emitted, not that Codex received it or a reviewer acted on it; interrupted delivery can still cause a duplicate. The optional Stop hook asks once per turn, and skips a continuation already started by a Stop hook.
codex_notice.py — translate a notice into Codex hook output
File: codex_notice.py
"""Optional Codex adapter; Python 3.10+, macOS/Linux."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import selectors
import signal
import sqlite3
import subprocess
import sys
import time
INPUT_LIMIT = 65536
OUTPUT_LIMIT = 65536
CONTEXT_LIMIT = 2400
def emit(value):
print(json.dumps(value, ensure_ascii=True), flush=True)
def run_notice(root, args):
process = subprocess.Popen(
[sys.executable, str(Path(__file__).with_name("spec_notice.py")),
"--baseline", args.baseline, "--map", args.map, "--json"],
cwd=root, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
start_new_session=True,
)
captured = {"stdout": bytearray(), "stderr": bytearray()}
deadline = time.monotonic() + 15
try:
with selectors.DefaultSelector() as selector:
selector.register(process.stdout, selectors.EVENT_READ, "stdout")
selector.register(process.stderr, selectors.EVENT_READ, "stderr")
while selector.get_map():
remaining = deadline - time.monotonic()
if remaining <= 0:
raise ValueError("notice command exceeded 15 seconds")
for key, _ in selector.select(min(remaining, 0.1)):
chunk = os.read(key.fileobj.fileno(), 8192)
if not chunk:
selector.unregister(key.fileobj)
continue
captured[key.data].extend(chunk)
if sum(map(len, captured.values())) > OUTPUT_LIMIT:
raise ValueError("notice output exceeded 64 KiB")
process.wait(timeout=max(0.001, deadline - time.monotonic()))
if process.returncode not in (0, 1):
error = (captured["stderr"] or captured["stdout"])[:1000].decode("utf-8", "replace")
raise ValueError("notice command failed: " + error)
finally:
# Also stop descendants retaining a pipe after their parent exits.
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
process.wait(timeout=2)
process.stdout.close()
process.stderr.close()
notice = json.loads(captured["stdout"])
if not isinstance(notice, dict) or not isinstance(notice.get("complete"), bool):
raise ValueError("notice output must contain complete: true or false")
for field in ("changed_paths", "affected", "unmatched"):
if not isinstance(notice.get(field), list):
raise ValueError("notice output requires a list: " + field)
if process.returncode == 1 and notice["complete"]:
raise ValueError("notice exit status contradicts complete output")
return notice
def output_for(event, message):
if event == "Stop":
return {"decision": "block", "reason": message}
return {"hookSpecificOutput": {
"hookEventName": "PostToolUse", "additionalContext": message}}
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--baseline", required=True)
parser.add_argument("--map", default="spec-map.json")
parser.add_argument("--state", required=True)
args = parser.parse_args()
raw = sys.stdin.buffer.read(INPUT_LIMIT + 1)
if len(raw) > INPUT_LIMIT:
raise ValueError("hook input exceeded 64 KiB")
event = json.loads(raw)
if not isinstance(event, dict):
raise ValueError("hook input must be an object")
name = event.get("hook_event_name")
if name not in ("PostToolUse", "Stop"):
raise ValueError("expected PostToolUse or Stop")
for field in ("session_id", "cwd"):
if not isinstance(event.get(field), str) or not 0 < len(event[field]) <= 4096:
raise ValueError("missing or invalid " + field)
if name == "Stop":
if not isinstance(event.get("stop_hook_active"), bool):
raise ValueError("Stop requires a boolean stop_hook_active")
if event["stop_hook_active"]:
emit({})
return
if not isinstance(event.get("turn_id"), str) or not 0 < len(event["turn_id"]) <= 256:
raise ValueError("Stop requires a bounded turn_id")
root = Path(subprocess.check_output(
["git", "rev-parse", "--show-toplevel"], cwd=event["cwd"],
timeout=5, text=True, stderr=subprocess.DEVNULL,
).strip()).resolve()
state = Path(args.state).expanduser().resolve()
if state == root or root in state.parents:
raise ValueError("state file must be outside the repository")
if not state.parent.is_dir():
raise ValueError("state directory does not exist")
notice = run_notice(root, args)
if (notice["complete"] and not notice["changed_paths"] and not notice["affected"]
and not notice["unmatched"] and not notice.get("warnings")):
emit({})
return
serialized = json.dumps(notice, sort_keys=True, ensure_ascii=True)
signature = hashlib.sha256(serialized.encode()).hexdigest()
key = hashlib.sha256((str(root) + "\0" + event["session_id"]).encode()).hexdigest()
turn = hashlib.sha256(event.get("turn_id", "").encode()).hexdigest()
# Preserve scope diagnostics when the path lists are truncated.
context = {field: notice[field] for field in ("complete", "scope_complete", "warnings")
if field in notice}
context.update(notice)
message = ("SDD review candidates, not a conformance result. "
"Check applicable requirements and current execution evidence.\n"
+ json.dumps(context, ensure_ascii=True))
if len(message) > CONTEXT_LIMIT:
message = message[:CONTEXT_LIMIT] + "\n[truncated; run spec_notice.py for the full report]"
# A private, explicitly selected database is delivery bookkeeping only.
old_umask = os.umask(0o077)
try:
connection = sqlite3.connect(state, timeout=2)
finally:
os.umask(old_umask)
with connection:
connection.execute("CREATE TABLE IF NOT EXISTS notices "
"(id TEXT PRIMARY KEY, signature TEXT, stopped TEXT, seen INTEGER)")
try:
connection.execute("BEGIN IMMEDIATE")
row = connection.execute("SELECT signature, stopped FROM notices WHERE id=?", (key,)).fetchone()
previous, stopped = row or ("", "")
duplicate = (previous == signature and notice.get("workspace_fingerprint_complete") is True
if name == "PostToolUse" else stopped == turn)
if duplicate:
emit({})
connection.rollback()
return
# Emission precedes committing suppression. A crash can repeat a notice.
emit(output_for(name, message))
connection.execute("INSERT OR REPLACE INTO notices VALUES (?, ?, ?, ?)",
(key, signature if name == "PostToolUse" else previous,
turn if name == "Stop" else stopped, time.time_ns()))
connection.execute("DELETE FROM notices WHERE id NOT IN "
"(SELECT id FROM notices ORDER BY seen DESC LIMIT 128)")
connection.commit()
finally:
connection.close()
if __name__ == "__main__":
try:
main()
except (OSError, ValueError, sqlite3.Error, subprocess.SubprocessError) as error:
print("SDD notice unavailable: " + str(error)[:1000], file=sys.stderr)
sys.exit(1)
The adapter bounds input, captured output and returned context, gives the notice command 15 seconds, and stops its process group on failure or completion. An error is visible as a failed hook; it does not certify the change or silently turn into an empty review. The scan still runs before duplicate suppression; an incomplete fingerprint is never used to suppress a post-tool notice. A complete scan with no changes or warnings stays quiet. The state file is a local convenience, not an authenticated audit record; remove it when the exercise is finished.
The following configuration is a fragment for a disposable project. Replace /absolute/external/state.sqlite3 with a writable path outside that project. Save it as .codex/hooks.json after examining the command and baseline. Existing hook sources are additive: deliberately merge the entry instead of replacing unrelated hooks.
PostToolUse configuration and optional Stop reminder
File: hooks.example.json
{
"hooks": {
"PostToolUse": [{
"matcher": "^(Bash|apply_patch)$",
"hooks": [{
"type": "command",
"command": "python3 \"$(git rev-parse --show-toplevel)/codex_notice.py\" --baseline review-base --state /absolute/external/state.sqlite3",
"timeout": 20,
"additionalContextLimit": 1000
}]
}]
}
}
To opt into a Stop reminder, add this entry alongside PostToolUse in hooks. This block requests another agent turn; it never grants or withholds acceptance, and complete is not an approval signal:
"Stop": [{
"hooks": [{
"type": "command",
"command": "python3 \"$(git rev-parse --show-toplevel)/codex_notice.py\" --baseline review-base --state /absolute/external/state.sqlite3",
"timeout": 20
}]
}]
Codex requires the project configuration layer and the exact non-managed hook definition to be trusted. Inspect and trust the fixture’s definition through /hooks; a new or changed definition can be listed but skipped until reviewed. The example does not bypass that decision. Inspect the referenced scripts too: trusting a command definition does not pin the contents of the repository-local Python file it runs. A stable handler requires a trusted copy or an independently checked script hash. Commands use the Python found on the host’s PATH; use an explicit interpreter path if the session and terminal environments differ.
The matcher covers supported shell and patch-tool calls, not arbitrary edits by another editor or every possible external writer. The optional Stop scan provides another opportunity to notice current differences; run the standalone check at review time regardless. Neither event establishes that all relevant consumers were mapped.
Give each skill a bounded responsibility
The six skills below share one contract model. A maintained capability contract contains settled requirements with stable IDs, applicability, observable outcomes, and one authoritative home per obligation. Scenarios and evidence link to those IDs. Task records carry unresolved decisions, investigation notes, and history; they do not become a second requirements authority. Code and tests show current behaviour, not automatically intended behaviour.
Run review skills with host permissions that enforce read-only access to the reviewed repository. The instructions alone provide no such boundary; the response checks below observed compliance but did not establish host-enforced isolation.
Route specification work to the smallest useful activity
File: skills/spectacular-spec-orchestrator/SKILL.md
---
name: spectacular-spec-orchestrator
description: Route specification work to the smallest bounded activity when a coding-agent task involves clarification, contract writing, specification review, behavioural examples, or implementation conformance.
---
# Spectacular specification orchestration
Choose the next consequential commitment. Do not force every task through every skill.
Use the maintained-contract model: settled requirements have stable IDs and one authoritative home; each requirement states its applicability and observable outcome. Keep unresolved decisions and history in the task or change record.
## Route by the work that is actually needed
- A report that an existing rule is broken: investigate the behaviour and repair it under the current contract. Do not invoke another specification skill merely because the report is a bug.
- A proposed implementation repair under an existing expected behaviour: select the applicable contract, then use `spectacular-conformance-review` to compare the implementation with it.
- A proposed change to expected behaviour: use `spectacular-spec-grilling` to settle the distinguishing product decision, then use `spectacular-spec-writing` for the authorised contract change.
- An expected outcome that the user or product has not decided: use `spectacular-spec-grilling` with a distinguishing case. If clarification concludes that the current contract already suffices, make no specification change.
- A question about ambiguity, authority, overlap, missing cases, or evidence: use `spectacular-spec-review` in read-only mode.
- A request to turn settled rules into acceptance examples: use `spectacular-bdd-gherkin`.
- A request to create or revise the maintained contract: use `spectacular-spec-writing`.
Inspect the request, applicable contract, change record, and available evidence before routing. State the capability, contract version, workspace or revision, next commitment, and the smallest useful handoff.
## Handoff
Return the selected activity, its bounded scope, the authoritative requirement IDs, evidence already available, limits or missing authority, and the next action. Re-evaluate routing after that activity produces a result; do not assume a fixed pipeline.
Keep review roles read-only through host permissions. This document cannot grant or enforce permissions.
Write a settled, bounded capability contract
File: skills/spectacular-spec-writing/SKILL.md
---
name: spectacular-spec-writing
description: Write or revise a bounded software capability contract with stable requirement IDs, one authoritative home per obligation, and traceable acceptance evidence.
---
# Spectacular specification writing
Write the smallest current contract that makes the next consequential commitment deliberate and reviewable.
## Establish scope and authority
Read the request, applicable contract, shared interfaces, recorded decisions, repository guidance, and relevant implementation evidence. Identify the capability, actor or trigger, supported version, affected consumers, and next commitment.
Classify inputs as an authorised decision, binding external contract, executed observation, user report, code-reading inference, hypothesis, or proposed design. Preserve source and version where they affect interpretation. Code and tests are evidence of behaviour; they do not silently become the intended contract.
## Write requirements
For each independently changeable obligation:
- assign a stable ID;
- state applicability: actor, input, state, trigger, time, or version;
- state the observable required outcome and relevant prohibited effects;
- define identity, units, cardinality, boundaries, and terms needed to interpret it;
- record rationale and source separately from the normative sentence;
- name acceptance evidence and its independent basis.
Keep one authoritative home for each obligation. A shared contract owns a genuine shared invariant or interface; a capability contract owns its local behaviour. References may point to that home but must not restate the rule. Preserve delegated implementation freedom when its bounds are clear.
Keep the maintained contract settled. Do not add lifecycle labels, transient workflow fields, or a material unresolved-decisions section. Put unanswered questions, hypotheses, and investigation history in the task or change record, and exclude dependent behaviour from the ready slice until its outcome is authorised.
## Reconcile before returning
Compare the proposed contract with active specifications, external constraints, code paths, and tests. Classify each mismatch as an implementation defect, specification defect, deliberate change, undocumented behaviour, stale secondary source, or unresolved conflict. Do not synchronise every source to the same mistake.
Return the exact bounded contract or patch, changed IDs and authoritative homes, source and basis, affected dependants, acceptance boundary, and any separate task decisions that remain. Recommend implementation only when the bounded slice has a settled outcome and a feasible verification path.
Resolve one consequential product decision with a concrete case
File: skills/spectacular-spec-grilling/SKILL.md
---
name: spectacular-spec-grilling
description: Resolve consequential software-requirement ambiguity through a focused interview with distinguishing cases and a durable decision record.
---
# Spectacular specification grilling
Clarify the next consequential product decision. The interview owns the decision record; the specification writer owns normative contract edits.
## Inspect first
Read the request, applicable contract, recorded decisions, relevant code and tests, and governing external sources. Verify facts directly. Classify uncertainty as a missing fact, a product choice, or an empirical question. If an authoritative source already answers it, report that answer instead of asking the user to rediscover it.
## Ask one useful question
Prioritise by consequence of a wrong answer, dependent work, reversibility, and available evidence. Ask one question per turn unless questions are genuinely independent. Use this shape:
Decision Q-<id>: <one concrete question>
Case: <small input, state, or timeline that separates plausible outcomes>
Why it matters: <observable consequence or blocked work>
Options: <materially different outcomes>
Recommendation: <only when evidence supports one>
Prefer an observable question such as “What should the caller receive after a retry?” over a quality adjective such as “Should retries be robust?”
Record the answer with its exact conditions, authority, basis, affected requirement IDs, examples, and dependent work. If the answer changes an existing promise, flag the reconciliation needed before contract text changes.
## Stop and hand off
Stop when the next bounded commitment has no unresolved consequential product decision, or every remaining uncertainty has an explicit disposition: investigate it, exclude dependent scope, defer independent work, or proceed within delegated discretion.
If the current contract already answers the case, return “no specification change” with the evidence. If a new or changed obligation is authorised, hand its settled wording and basis to `spectacular-spec-writing`. Keep questions and history in the decision record, never in the maintained contract.
Audit contract meaning, authority, interaction, and evidence
File: skills/spectacular-spec-review/SKILL.md
---
name: spectacular-spec-review
description: Audit a bounded software contract for ambiguity, contradiction, duplicate authority, missing cases, weak evidence, and drift without treating code as the intended authority.
---
# Spectacular specification review
Review the selected applicable contract and its justification. This is a read-only quality and reconciliation review, not a code-authority vote and not an implementation approval.
## Establish the review basis
Identify the capability, contract version and applicability, shared contracts, authorised decisions, relevant revision or workspace state, acceptance examples, tests, and binding external interfaces. Inspect sources directly and label each observation as normative contract, decision record, code behaviour, test expectation, runtime observation, or external contract.
## Review passes
1. **Meaning and authority:** find the maintained home, applicability, required outcome, prohibited effects, and basis for every consequential obligation. Use a concrete witness where two behaviours could fit the wording.
2. **Overlap and interaction:** classify related rules as duplicate authority, contradiction, partial overlap, legitimate refinement, shared applicability, or disjoint cases. Check combinations and lifecycle boundaries, not only pairs.
3. **Consequential gaps:** probe only boundaries that can change acceptance for this increment: identity, permissions, limits, retries, cancellation, partial failure, ordering, concurrency, time, preservation, and version coexistence.
4. **Acceptance and evidence:** trace important rules to evidence and evidence back to rules. Separate proposed checks from executed results; state the needed fixture, measurement, or counterexample.
5. **Reconciliation:** compare changed decisions with active contracts, external constraints, code paths, and tests. Code describes current behaviour, not automatically intended behaviour. A test records an encoded expectation, not automatically the governing rule.
Classify mismatches as implementation defect, specification defect, deliberate change, undocumented behaviour, stale secondary source, or unresolved authority. Recommend the smallest correction at the source that is wrong. Do not edit requirements or tests merely to make them agree with code. When authoritative rules conflict without precedence, report the incompatible outcomes and the decision needed from their owner. Do not select a product policy, weaken an obligation or invent an exception to fit existing code. Distinguish an uninspected boundary from an observed defect; an isolated function does not establish what its callers validate.
## Report
Order material findings by consequence. For each, give location and requirement ID, classification, concrete witness or evidence gap, consequence, smallest repair or decision, and affected dependants. State the inspected boundary, inaccessible sources, and any work that remains unverified. A structurally clean document is supporting evidence only; it does not establish semantic correctness.
Keep this role read-only through host permissions. The skill instructions alone cannot enforce that boundary.
Turn settled requirements into traceable behavioural examples
File: skills/spectacular-bdd-gherkin/SKILL.md
---
name: spectacular-bdd-gherkin
description: Discover, write, or review Gherkin acceptance examples from settled requirements while keeping expected results traceable to an independent basis.
---
# Spectacular BDD with Gherkin
Use examples to sharpen and verify behaviour. The maintained contract remains the authority; a scenario illustrates or checks a rule and does not create a parallel rule.
## Select and ground the rule
Choose a settled requirement ID and its applicability. Select a distinguishing example that separates plausible interpretations or exercises a consequential boundary. Link the expected result to an authorised decision, binding external contract, or independently justified property. Record fixtures, environment, and verification boundary separately.
Write observable Gherkin with one material event:
Feature: <capability>
Rule: <REQ-ID> <short behavioural rule>
Scenario: <distinguishing outcome>
Given <relevant domain state>
When <one material event>
Then <observable outcome>
And <observable invariant when relevant>
Use `Scenario Outline` only when data varies under the same rule. Use `Background` only when shared state stays clear in every scenario.
## Discovery and review
During discovery, label examples as rule, example, question, or deferred behaviour. Alternative outcomes are hypotheses for discussion, not canonical expectations. When the oracle is missing, preserve the concrete case in the task record and route it to `spectacular-spec-grilling`; do not guess a canonical `Then` outcome.
For each canonical scenario, verify its rule link, observable boundary, independent expected-result basis, distinct behavioural value, and consistency with active contracts and deliberate changes. Distinguish an expected result from an executed result. When an existing scenario disagrees with the contract, classify the mismatch before changing either side.
Return scenarios, requirement and contract-version traceability, expected-result basis, fixtures, verification boundary, and clearly labelled hypothetical cases. Keep unresolved questions out of the maintained contract.
Compare an implementation with its selected contract and actual evidence
File: skills/spectacular-conformance-review/SKILL.md
---
name: spectacular-conformance-review
description: Perform a read-only conformance review of an implementation against selected applicable requirements and actual execution evidence, reporting supported, violated, or unverified outcomes.
---
# Spectacular specification conformance review
Compare the implementation with the selected applicable contract. The contract establishes intended behaviour; code-reading inference, tests, and executed runs provide different kinds of evidence about current behaviour. Do not silently change either to make the comparison pass.
## Fix the review basis
Record the capability, contract version and applicability, requirement IDs, code revision or dirty workspace, relevant consumers and entry points, configuration and fixtures, commands actually run, results, and evidence limits. Label each observation as code-reading inference, test expectation, executed observation, or runtime observation. Re-check the selected scope after relevant edits or intervening merges. When claiming that a test or command executed, retain its actual argument vector and result; a passing command is not evidence if its oracle is weak or its path is unexercised.
Trace each requirement through all relevant implementation paths, including alternate entry points and shared dependencies. Use the evidence suitable for the claim: meaningful inspection can support an inspection-level conclusion, while execution claims require an actual run record. Not every requirement needs a command run. Treat evidence as stale when the code, fixture, configuration, or applicable revision changed after it was produced.
## Report three outcomes
For each selected requirement, report exactly one evidence outcome:
- **supported:** observed behaviour and credible evidence satisfy the applicable obligation;
- **violated:** an observed path or result conflicts with the obligation;
- **unverified:** the available evidence cannot establish either conclusion.
Separate “inputs matched the recorded run” from “the recorded command passed.” Include actual command arguments, result, bounded logs, workspace or revision fingerprint, and limitations. A test that always returns the expected value, an unexamined alternate path, or a skipped/untrusted hook does not establish conformance.
## Route the finding
Do not edit requirements, implementation, or tests as part of this review. A violated current rule is an implementation repair. If the contract appears wrong, route the concrete conflict to `spectacular-spec-review` and, after an authorised decision, `spectacular-spec-writing`. If the expected outcome is not settled, route it to `spectacular-spec-grilling`. If evidence is insufficient, name the smallest investigation or run needed.
Return the selected scope, per-requirement outcome, concrete witness or evidence gap, executed evidence record, limitations, affected dependants, and next action. Keep review authority and implementation authority separate; this report is not authenticated approval.
Keep the role read-only through host permissions. Skill text cannot enforce permissions or undo a completed action.
Decide whether the additional work earns its cost
Choose a change and record the starting point
Use the worksheet to make a local decision, not to manufacture a productivity number. Compare the proposed structured review with the team’s existing practice first. To evaluate notices, hold that review procedure constant and compare it with and without notices; otherwise the experiment changes two things at once.
Scroll horizontally to see all columns.
| Record before the trial | Record for every included change |
|---|---|
| Capability, change types, inclusion rule and observation window. | Included, excluded and unfinished changes, with reasons. Keep failed and abandoned attempts visible. |
| What counts as ambiguity-related rework: work repeated because an expected outcome was missing, disputed or misunderstood. | The concrete decision and repeated work; distinguish code defects and unrelated redesign. |
| What counts as useful feedback: an observation that changes or confirms the next decision. | Time from starting preparation to that observation, including waiting for clarification. |
| One-time setup and teardown, reported separately from recurring work. | Map preparation, script adaptation, hook configuration/trust, reviewer calibration, trial analysis and removal or migration of trial files/state. |
| Existing review procedure and the single addition being evaluated. | Human preparation, implementation, review, correction and subsequent contract/history maintenance effort; material model and tool costs. |
| Local retain, simplify or drop criteria, chosen by the responsible team. | Useful findings, missed obligations found later, false or duplicate notices, scan time and review time. |
| How changes will be assigned and differences recorded. | Difficulty, reviewer, model/client version, environment and missing evidence. |
A fixed window or preselected sequence makes convenient examples harder to cherry-pick. Alternating the added practice across comparable changes may reduce some selection effects; it does not make different changes equivalent. A comparison with recent history is observational: learning, task mix and tool changes can explain a difference. Keep those limits attached to the result.
Do not infer missed-defect rates from notices alone. An unchanged independent review or later incident may reveal an obligation both workflows missed. A local decision can still use incomplete evidence, provided it states what was observed and what judgement carries the remaining uncertainty.
Specify, implement and review one increment
Choose a bounded change with an agreed outcome and existing review responsibility. Use the current contract, the relevant implementation paths and the checks that can establish its behaviour. Record actual results and unresolved limits in the existing task or PR. For the notice comparison, keep that review unchanged; introduce only the notice and measure what it adds or misses.
Examine the cost evidence
The recommendations here remain engineering judgements, not a complete method validated by the studies below. The evidence can identify costs that a local trial should measure; it cannot establish that this proposed workflow earns them for a particular team or feature.
Eberhardt’s 2025 Spec Kit experiment reported 3.5 hours of review for a circuit-management increment and about two hours for a GPS addition, with much of that time spent reviewing Markdown. This is one developer’s account of one application with the tools available then. Review time belongs in the local comparison because it is part of the workflow’s cost.
Spec Kit Agents, April 2026 preprint
Spec Kit Agents compared a full agent workflow with a version adding grounding and validation. Across 32 tasks in five repositories, the quality score, weighted by feature count, rose from 3.51 to 3.66 on a 1–5 scale. Another language model judged qualities such as correctness and maintainability. That score is an evaluation proxy, not direct evidence of user value.
Among 16 pairs of completed runs under a 90-minute budget, mean latency rose from 24.0 to 37.2 minutes. The latency analysis excluded incomplete and rate-limited runs. Rate-limited runs could still contribute quality scores when a pull request was available.
Human reviewers subsequently compared pull requests produced with and without the additions. Across six tasks, their 60 judgements included 33 ties, 19 preferences for the workflow without the additions and eight for the augmented workflow.
During agent execution, plan-review checkpoints were automatically approved; the human preference review happened afterwards. The study therefore cannot establish the value of human review during development. The comparison is small and leaves the additions’ value for a particular team unresolved. Measure the trade-off locally, including the time spent on the whole workflow.
Decide which parts earned their cost
Compare the observations with the team’s preselected criteria. Keep an addition when its useful findings or reduction in rework justify its preparation, maintenance and feedback delay. Simplify or remove checks that mainly produce noise. If missing history or different task difficulty prevents a useful comparison, report that instead of a reduction percentage.
The runnable fixtures establish behaviour of the example scripts and selected skill responses. They do not establish a team productivity gain or validate the hypothetical invoice-export investigation.
Fixture and response checks
The published blocks were extracted into temporary files and exercised directly. Structural checks rejected duplicate or empty definitions and broken references. Git fixtures covered dirty files, renamed paths, shared consumers, deleted relationships and a deleted map. Recorder checks distinguished a failed command with matching inputs from a passing command whose inputs changed, including dependency and permission changes. Timeouts, bounded logs, malformed input and incomplete scans were exercised separately.
The actual fixture record in Part 6 includes an ordinary repair and a deliberately changed requirement. An API-only check passed while an alternate download path remained broken; the full fixture suite detected that path. This establishes sensitivity of those checks, not correctness of a real authorisation service.
Six independent agent sessions received one skill and a bounded fixture, with the expected evaluation answers withheld. The trials covered repair routing, settled writing, clarification answered by an existing rule, contradictory requirements, BDD with an undecided survivor rule, and conformance with an untested entry point. The first specification reviewer chose a policy while reporting the conflict. The instruction was tightened to return that choice to its owner; a fresh reviewer did so. These few responses do not establish repeatability across models. The exact model identifier was not captured for those six skill trials, and read-only behaviour was observed rather than enforced by an isolated host.
The native Codex fixture used the published adapter with normal /hooks trust. Post-tool context reached the agent; the first Stop requested continuation and the guarded next Stop completed without a loop. A preliminary probe also observed untrusted hooks being skipped and a malformed Stop response being rejected. Temporary trust entries were removed afterwards. Native event delivery was checked separately from direct script execution; the notice’s final scan-deadline and report-limit changes were then checked directly. None of these observations establishes a merge gate or authenticated approval.
Kiro, Spec Kit and OpenSpec were compared through documentation only. No team productivity trial or invoice-export experiment was run. The worksheet above defines the observations a local cost comparison would still need.