ADR-0007.

Date: 2026-07-07 Status: accepted Deciders: Thibault Souris (maintainer) Method: decision-loop — reality-check against the competitor benchmark (BMAD's adversarial review), challenge, durable position

Context

The competitor benchmark surfaced the one place a competitor is genuinely ahead: BMAD's LLM evaluation layer — adversarial multi-subagent review, "run it on a different model", deduplicated triage — is a maturity runward lacks. The deterministic gate check --strict (ADR-0001) verifies the presence of a traced decision (applied with a pointer, deviated with an ADR, n/a with a reason). What it cannot judge is semantic: does the code an applied row points at actually apply the rule, or merely cite it? That is exactly the incident that started this whole line of work, and it is a judgment a deterministic check cannot make.

An LLM can help there. But ADR-0001 fixes the invariant: no LLM in the gate path. And the benchmark named the real danger — an "advisory" LLM opinion becoming the de-facto gate by convenience, which would reproduce the very incident (a machine validating instead of a deterministic gate an operator crosses).

Decision

Introduce an advisory LLM conformance-verification layer as a workflow (verify), agent-executed and strictly subordinate to the deterministic gate:

• Agent-executed, nothing bundled. verify is a markdown workflow the operator's own coding agent runs — no LLM in runward's code, vendor-neutral, like every other workflow.
• Never in the exit-code path. It is not wired into check, produces no exit code, and never blocks. It emits cite-vs-apply findings (confirmed / cited-not-applied / uncertain) for the operator to read.
• Adversarial and diverse. It challenges each applied claim through distinct lenses and, where possible, a different model than the one that built the floor — to avoid self-agreement.
• The gate stays the operator's. The deterministic check --strict remains the load-bearing decision; verify only informs the operator's judgment before they cross it.

The four structural guards — separate workflow, opt-in, advisory-only, gate unchanged — are what keep it from drifting into the gate.

Alternatives discarded

• Wire the LLM into check --strict. Violates the zero-LLM invariant and makes the advisory the gate — the exact failure mode named in the benchmark.
• No LLM layer at all. Leaves the semantic cite-vs-apply gap that no deterministic check can close; the benchmark showed this is where a competitor leads.
• Bundle an LLM provider in runward. Breaks vendor-neutrality; the workflow-executed-by-the-agent model keeps runward neutral.

Consequences

• Positive. The semantic gap (cited vs applied) gets a real instrument, without touching the deterministic gate or vendor-neutrality. runward closes its one benchmarked weakness — on its own terms.
• Negative, accepted. An advisory layer can be misused as a gate; the workflow and this ADR state, explicitly and repeatedly, that it never is. The operator must resist rubber-stamping its verdict.
• On other boundaries. A new verify workflow and /rw-verify command; check is untouched; the zero-LLM invariant of the gate is untouched.

Reevaluation trigger (mandatory, dated)

Reopen if operators begin treating a green verify as the pass — the advisory becoming the gate. The response is to re-assert subordination and, if needed, make the coupling looser, never to wire verify into the exit code.

Trigger set on: 2026-07-07 · Watched via: how teams describe their gate-crossing decision in handover.

Amendment (2026-07-11) — discoverability wired, /rw-verify left per-harness

The verify workflow shipped with the decision (templates/workflows/verify.md), but nothing surfaced it: an operator crossing a green gate never learned the semantic layer existed. Fixed, without touching the invariant. (1) runward check --strict, when green, now prints an advisory "Semantic check" pointer to runward/workflows/verify.md (agent-executed, never blocks). (2) AGENTS.md instructs any agent to run verify before crossing a green gate. Both are zero-LLM, vendor-neutral prose — the human and the agent now both find the layer at the right moment.

The /rw-verify slash command named in Consequences is deliberately not shipped as a default: a per-harness slash command would privilege one agent, against the vendor-neutral invariant (ADR-0010/0012, later than this ADR). The neutral surface — the workflow plus the two pointers above — carries it; a /rw-verify binding stays a per-harness example the operator may add, like the Claude Code Stop hook.

Amendment (2026-07-16) — the findings become an artifact the gate can see

The audit asked for the semantic pass to be promoted into an executable opt-in (check --verify). Wiring an LLM into runward's code stays refused — that is this ADR's founding line. What was missing is deterministic: the workflow handed its findings to the operator as conversation, leaving no artifact, so a CI or a handover could not even tell whether the pass ever ran. Fixed at the artifact level: the verify workflow now writes its findings to runward/governance/verify-findings.md (dated, one finding per row), and a green check --strict surfaces that artifact's presence and freshness — fresh when newer than every gated manifest, stale when a manifest changed since — exactly the behavioral-proof pattern. The gate never parses a verdict out of it and never blocks on it: presence and date are bytes; the judgment stays the operator's. The opt-in executable the audit wanted exists now as the operator's agent running the workflow, with a trace the pipeline can see.

References

• ADR-0001 — the zero-LLM gate this sits above, never inside.
• BMAD bmad-code-review (adversarial multi-subagent review, "different LLM", triage) — the reference for the advisory layer.
• templates/workflows/verify.md, templates/workflows/review.md — the workflow surface.