ADR-0114
ADR-0114-an-identifier-is-read-outside-comments-and-a-strict-green-requires-a-row-the-gate-verified.md
ADR-0114 — An identifier is read outside comments, and a strict green requires a row the gate verified
Date: 2026-11-05
Status: accepted (2026-11-05)
Deciders: an agent delegate, under class D of runward/delegation.md, on the maintainer's instruction
of 2026-11-05. Under ADR-0088 decision 4 the agent's writing is disclosed throughput, not independent
review; the pull request that carries this record changes pre-merge paths and waits for the maintainer
Method: both defects reproduced on a fresh clone of stranxik/runward at main@8ffdf7c (0.46.0)
before any change, each beside its control, then re-measured on the change. Read: src/lib/evidence.ts
(symbolPresent and its callers), src/lib/conformance.ts (PATH_TOKEN), src/lib/verdict.ts,
src/lib/check-contract.ts, src/commands/check.ts, src/commands/verify.ts, docs/spec/witness.md,
runward/contracts/port-contract.md, action.yml, ADR-0019, ADR-0051, ADR-0053, ADR-0060, ADR-0083
Relates to: ADR-0019 and
ADR-0051 decision 1 (the symbol depth, refined
by decision 1 here), ADR-0060 (the vacuous green,
refined by decision 2: the refusal it described), ADR-0030
and ADR-0083 (the exit
code contract, which gains 5), ADR-0053
(a prefix verdict), ADR-0054 (nothing is executed),
RWD-2026-0194 in known-defects.md
Context
A comment kept a renamed symbol green (RWD-2026-0194). The README's two-minute demo renames
guardFields to validateFields throughout code/src, and check --strict exits 1 on the two rows that
still cite file:code/src/core/domain/guard.ts#guardFields. Measured on 2026-11-05: append one line,
// validateFields was called guardFields before the rename., and the same command exits 0, "all gates
passed", over a floor note and a threat model that cite a function that no longer exists; the same with a
/* … */ comment. The check was the one ADR-0019 decided, "the symbol occurs in the file (deterministic
substring, language-neutral)", at the identifier boundary ADR-0051 added. A comment holds the name.
Where runward resolves a symbol. Three readers search a whole file: the gate (evidenceReport),
spec-check (specConformance) and the rules --for reminder (citationsForPaths). Two locators read
line by line: the witness's symbolAt and the ratification excerpt. Each reads any file that is not a
report an adapter routes (SARIF, ESLint, CycloneDX, k6 or JMeter, LCOV or Cobertura). runward has no
notion of a language for a symbol; the languages it knows by extension are the evidence layer's
PATH_TOKEN: ts tsx js jsx mjs cjs py md json yml yaml toml go rs java rb php sql sh css scss html txt.
Measured on 2026-11-05, the 52 #SYMBOL pointers of runward's own mission and of the shipped example sit
in .md (20), .ts (18), .json (8), .yml (4) and .js (2) files.
A mission where nothing was verified exited 0 (case c4 of the 2026-11-05 witness). The example
mission, every one of its 46 manifest rows set to n/a with one generic sentence: check --strict prints
0 applied · 0 deviated · 46 n/a, "! nothing was verified mechanically", then "all gates passed", exit 0;
--json says verdict: "clean", exitCode: 0. ADR-0060 decided that a vacuous green is disclosed, not
refused, and wrote what a refusal would be if one came: "it takes Jest's shape: a named opt-in flag, never
a silent pass". Its precedent list already held the exit code: pytest exits 5 when no test was collected.
What the exit code contract says. runward/contracts/port-contract.md publishes 0 clean, 1 gaps, 2
"the question could not be asked", and its evolution rule: "existing exit-code semantics ... never change
meaning"; a breaking change "would ship alongside the old shape". No runward command exits 5 today
(grep of src/ on 2026-11-05).
Options considered
For the symbol:
- A. Remove the comments, by a lexer per known language, before the search; keep the strings. A file of another language is read whole, as before.
- B. Remove the comments and the strings. runward's own architecture note cites
file:eslint.config.js#no-restricted-imports, a name that lives only in string literals, as do config keys, routes and rule ids everywhere: B reds honest evidence. - C. Resolve symbols by syntax tree or language server. ADR-0019 discarded it: language machinery, dependencies, false precision.
- D. Keep the textual reading and say so (RWD-2026-0194's workaround: grep the old name after a rename).
For the vacuous green:
- A. Under
--strict, a level of proof is required;mechanicalby default, refused with exit 5;--require tracedaccepts by name. Jest's shape, pytest's code. - B. Refuse with exit 1. It says "gaps", and the payload's counts (RWD-2026-0146: what stands between a run and a green) would hold a gap that is no row.
- C. Keep the disclosure (ADR-0060 as it stands): the case c4 green stays the default.
- D. An opt-in refusal, the default unchanged. The vacuous green stays the default, and the operator who needs the refusal is the one who never asks for it.
- E. A deprecation window: warn for one release, refuse in the next. The contract's "expand then contract"; the default stays vacuous for the length of the window.
Decision
A for both, on the maintainer's instruction of 2026-11-05.
1. A #SYMBOL of identifier form is looked for outside the file's comments. withoutComments
(src/lib/evidence.ts) replaces every comment of a file by spaces, its line breaks kept, so no two tokens
are glued and every line keeps its number; symbolPresent(content, symbol, path) searches what remains
for a symbol of the form ADR-0051 matches at an identifier boundary (^[A-Za-z_$][A-Za-z0-9_$]*$). The
three whole-file readers pass the path; the two locators do not, and keep the text.
- The languages: those of
PATH_TOKEN, plus.mts,.ctsand.markdown. JavaScript and TypeScript (//,/* */; strings, template literals with${…}, regular-expression literals after an operator or a keyword), Python (#; triple-quoted strings), Markdown and HTML (<!-- -->; in Markdown a backtick run opens code that a run of the same length closes), JSON (//,/* */, as tsconfig and JSONC write them), YAML (#at the start of a word; quotes open a string where a value starts; block scalars are strings), TOML (#), Go (//,/* */; raw strings, runes), Rust (//, nested/* */; raw strings, characters, lifetimes), Java (//,/* */; text blocks), Ruby (#,=begin…=end), PHP (//,#except#[,/* */), SQL (--,/* */), shell (#at the start of a word), CSS (/* */), SCSS (//,/* */)..txthas no comment, and any other file is read whole, as before this record. - Strings, decided: a string literal is code that ships and runs, so a name in a string counts, and a
comment marker in a string (
"https://…",'#','a--b') opens nothing. A Python docstring is a string: it counts (stated below as a limit). - Other symbols, decided: a quoted sentence, a dotted name or an operator keeps the textual reading.
It names no identifier, ADR-0051 already reads it as text, and a sentence's natural home is a comment or
a document: runward's own suite cites
#"the guard fails closed"from a code comment (test/unit/evidence-bytes-and-boundaries.test.js). - An unterminated comment runs to the end of the file in a programming language (what follows it is not
code), and is text in Markdown and HTML, where an unclosed
<!--in prose is literal. - The refusal keeps its sentence, and adds "— it occurs only in a comment, and a comment is not code (runward ADR-0114)" when the file's text holds the name, so the reader is not told "not found" over a file that visibly contains it.
- The witness (runward-witness/1) keeps
symbolAtas section 9.5 defines it, on the text: a refusal of a symbol present only in a comment is beyond the witness, familycomments.
2. Under --strict, the verdict requires a level of proof. check --strict --require <level>:
mechanical, the default of a full strict verdict: at least oneappliedrow carries a pointer the gate opened and checked (evidence.typed > 0, the count the run already prints).traced: a green that rests on judgment alone, accepted by name. A--throughprefix requirestracedunless--requiresays otherwise (ADR-0053: a horizon certifies progress, never a crossing).--requirewithout--strictis a usage error, exit 2: no row is read.- Exit 5 when no gap stands and the level is not met: the run was asked and answered, so not 2; it found no gap, so not 1; nothing it verified reaches the level, so not 0. pytest's code for "no tests were collected" carries the same distinction, and 5 is free in runward's contract. Gaps first: a red verdict exits 1 whatever its level.
- The summary says it: "Current gate deliverables filled, nothing verified mechanically", a closing
line naming the level, the count and both ways out, and a Next line (
verify-a-row) that names them. Accepted withtraced, the run still prints that the green rests on judgment alone. --jsoncarries it:verdict: "level-not-met",exitCode: 5, and, under--strict,require: { level, met }.verdictOutcome(src/lib/verdict.ts) is the one reading of the words and the code.- Every surface that states the verdict states this one. The attestation carries it (its predicate is
the payload).
verifyre-derives at the level an attestation records; one that records none was written by a runward that required none and is re-derived attraced, whichverifysays on a line of its own. The VSA reads FAILED; the JUnit rendering'srunward gatecase fails,runward.exit-code5;report(HTML and Markdown) reads LEVEL NOT MET;statusreads "no gap, nothing verified mechanically"; the compliance pack's banner says "level not met" and a control isimplementedonly on exit 0;--freezerefuses to seal, the exit staying 5. The witness keeps its arithmetic (exitCode0 or 1) and gains the memberverdict.require, carried, not re-checked (familyrequired-level). SARIF and Code Quality list findings at locations, and a level is not one: they gain none, and the job's exit code carries it. - The port contract gains the row 5. The old green stays one flag away, which is the "alongside" of the contract's evolution rule; there is no deprecation window (option E), because a window keeps the vacuous green the default for its length.
How it extends: levels of proof
A study of 2026-11-05 proposes three levels of proof per row, re-derived by runward from the tree, attested (produced out of the agent's reach), declared, and a level a run demands; it is the subject of a later record (ADR-0119, for the maintainer to ratify). This decision is its first step, in its shape:
--require <level>names the floor a run demands.tracedadmits rows that are all declared;mechanicaldemands one re-derived row. A later level slots intoREQUIRED_LEVELSby rank (they are ordered from the least to the most demanding), andrequire.metand exit 5 keep their meaning: the verdict meets the level asked, or no gap stands and it does not.- The level of each row is not named in
--jsonyet. It is derivable today: anappliedrow listed inevidence.proseRowsis declared, every otherappliedrow was re-derived. Naming it waits for ADR-0119's vocabulary, because ADR-0030 forbids renaming a field once it is emitted. - Whether
mechanicalstays "one row" or becomes a floor every row must reach is ADR-0119's question.
Alternatives discarded
- A comment-stripping regular expression that ignores strings.
"https://…"holds a//and'/*'a/*: it would cut the rest of such a line, and refuse an honest symbol that follows a URL on it. - An exit code above 100, or 3 or 4. No precedent carries the meaning; 5 does, in the tool ADR-0060 already cited.
- A boolean flag (
--allow-vacuous,--pass-with-nothing-verified). It names one exception, where a level names a scale the next record extends.
Consequences
- The README's demo stays red with the old name in a comment (
test/unit/symbols-outside-comments.test.js), and case c4 exits 5 (test/unit/required-level.test.js). Runward's own mission and the shipped example stay at exit 0,require: { level: "mechanical", met: true }; an attestation of 0.46.0, of the example or of case c4, still verifies. - Two behaviours change for adopters (CHANGELOG, Changed): a pointer whose name survives only in a
comment turns red, and a strict mission none of whose
appliedrows the gate verified exits 5 until a row is pointed at what applies it or the run says--require traced. - The lexer is a lexer, not a parser. Its limits, each rare: shell, Ruby and PHP heredocs, JSX text,
a regular expression after
)or after a keyword not in its list, SCSS's unquotedurl(//…), Ruby's?#and a Ruby regular expression holding a#, MySQL's#comments, the bodies of HTML<script>and<style>, Markdown~~~fences and indented code. Each misreads a span either as code, so a comment there still counts (the reading before this record), or as a comment, so a name there is refused (red, and said). - Seven modules of the mutation perimeter change (evidence, verdict, check-contract, spec-conformance,
citations, witness, compliance):
npm run mutation:freshnames them stale until they are re-measured, before the next tag (ADR-0103). - Unchanged. The turn-end
gate-hookblocks on gaps only. The Action and the shipped CI samples pass the code ofcheck --strictthrough, so 5 fails their job as 1 does, and the pre-commit sample refuses the commit; their comments still name 0, 1 and 2 only, and naming 5 there is left to the next adapter sync. The Claude Code hook sample stays advisory (|| true), and the BMAD review layer reports the code verbatim. A signature (ADR-0020) and a test name (::NAME) in a source file are still read on the text, comments included.
What this does not catch
- A name left in a string or a docstring, decided above, and a real identifier unrelated to the rule (ADR-0051, "What this does not claim").
- One honest pointer and everything else declared meets
mechanical; a level per row is ADR-0119's. - A file outside the table, read whole as before.
What would settle it
- For A (symbols): over the next ten pull requests that touch a cited file of runward's own mission or of an adopter's that reports to the register, no refusal comes from a comment marker the lexer misread.
- Against A (symbols): one confirmed refusal of legitimate evidence caused by the lexer.
- For A (level): every vacuous mission reported after the release exits 5, and the adopters who keep a
traced mission say
--require tracedin their CI, visibly. - Against A (level): a legitimate mission that can point no row at anything, refused in a way that sends it to change a correct layout rather than to name its level.
Reevaluation trigger (mandatory, dated)
Reopen at the first confirmed false refusal under decision 1, at the first report described under "Against A (level)", when ADR-0119 is decided, or on 2027-02-05, whichever comes first.
Trigger set on: 2026-11-05 · Watched via: test/unit/symbols-outside-comments.test.js,
test/unit/required-level.test.js, the known-defects register, and issues that mention exit 5 or
--require.
References
src/lib/evidence.ts(withoutComments,symbolPresent,COMMENT_ONLY),src/lib/spec-conformance.ts,src/lib/citations.ts.src/lib/verdict.ts(RequiredLevel,requiredLevel,levelMet,verdictOutcome,LEVEL_NOT_MET_EXIT),src/lib/check-contract.ts,src/commands/check.ts,src/commands/verify.ts,src/lib/witness.ts,docs/spec/witness.md(6.2, 8.2),runward/contracts/port-contract.md.- pytest exit codes (5: no tests collected); Jest
--passWithNoTests; ADR-0060's prior art. - ADR-0019, ADR-0051, ADR-0053, ADR-0060, ADR-0030, ADR-0083, ADR-0103, ADR-0088 decision 4; RWD-2026-0194.