ADR-0116
ADR-0116-what-a-user-installs-is-what-the-release-tested-the-tarball-bundles-its-dependencies-the-sbom-is-the-packages-and-no-signing-job-runs-the-build.md
ADR-0116 — What a user installs is what the release tested: the tarball bundles its runtime dependencies, the SBOM describes the package, and no job that signs runs the build
Date: 2026-11-05
Status: accepted (2026-11-05) — the release tarball carries its runtime dependencies at the versions
of the tag's lockfile (bundleDependencies), the release SBOM is the package's own (CycloneDX 1.6,
derived from the tarball and the lockfile, re-derivable by anyone), the jobs that hold id-token: write
in the release chain check out nothing and run no code of the tree or its dependencies, a release whose
runward-forge is not yet on npm publishes runward, warns and stays green, Node.js 26 joins the unit
test matrix, the OSCAL ingestion installs from a hash-pinned closure, and the CI job that writes the
reference JUnit report keeps it as an artifact
Deciders: delegate agent, class D of runward/delegation.md, on the maintainer's instruction of
2026-11-05. The change touches .github/** and package.json, pre-merge paths of the charter: the
pull request waits for the maintainer before merge (ADR-0088 decision 7). Under ADR-0088 decision 4 the
maintainer is the single accountable person, and the agent's writing is disclosed throughput, not
independent review
Method: measured on 2026-11-05 on a fresh clone of stranxik/runward at main@8ffdf7c (0.46.0),
macOS, Node.js 24.22.1, npm 11.19.1, every install into an empty project with its own npm cache and an
empty user configuration: the tarball of main installed as it is; the same tarball with a runtime-only
npm-shrinkwrap.json (derived from package-lock.json, 28 entries, 15,058 bytes), installed from the
file and from a one-package local registry that serves it with _hasShrinkwrap set to true, then to
false, every other request redirected to registry.npmjs.org; the tarball packed with
bundleDependencies, installed from the file online, then offline with an empty cache and an unreachable
registry; npm audit signatures on the bundled and unbundled installs; on the committed change, the
guide's Step 10 replayed on a tarball packed from it (cmp equal) and that tarball installed from the
file, offline and from a local registry (27 of 27 each). Read: the v0.46.0 release assets
(runward-sbom.cdx.json), .github/workflows/build-and-attest.yml, release.yml, verify-release.yml,
ci.yml, the npm 11.19.1 and 12.2.2 sources (@npmcli/arborist, npm-packlist, lib/utils/verify-signatures.js)
and npm 12.2.2's docs/content/configuring-npm/package-lock-json.md, the registry's package documents of
runward, sfdx-cli, @salesforce/cli and firebase-tools, nodejs.org/dist/index.json and
github.com/nodejs/Release schedule.json, slsa.dev/spec/v1.2 (Build track basics and requirements), the
GitHub API (release and verify-release runs, the failed step's log of run 40684642168, the open
code-scanning alerts, all read with GET), and cmark-gfm's table scanner (extensions/ext_scanners.re,
extensions/table.c)
Relates to: ADR-0031 (the supply-chain evidence a regulated buyer reads),
ADR-0048,
ADR-0049 (the isolated builder, refined here),
ADR-0062 (an emitted document is
validated against the official schema),
ADR-0085,
ADR-0094
(runward-forge),
ADR-0110,
ADR-0112
(the reference environment, refined by decision 9)
Context
What a user ran was not what the release tested. The tarball declared three dependencies by range and
carried no lock. Installed into an empty project, the 0.46.0 tarball resolved 16 of its 27 runtime
packages to other versions than the tag's package-lock.json (every @inquirer/* package, by a patch
release each); the suite, the self-gate and the SBOM had all used the lock's versions. For a tool whose
promise is the same verdict on the same tree, the code that runs at an adopter's was neither the tested
code nor the code the SBOM listed.
What a shrinkwrap would and would not have fixed. A runtime-only npm-shrinkwrap.json in the same
tarball changed nothing for an install from the file: 11 of 27 again. npm reads a dependency's shrinkwrap
only when the registry's metadata marks the version _hasShrinkwrap: true (node.hasShrinkwrap in
@npmcli/arborist), which an install from a file never has. Served by a local registry with the flag
true, the same tarball installed 27 of 27; with the flag false, 11 of 27. npmjs.org sets the flag
from the tarball (it is true on sfdx-cli 7.209.6 and on every @salesforce/cli version published
with npm 10 and 11, false on runward 0.46.0, and the npm 11 client writes no such field: an inference,
since the registry's code is not public). And npm 12 (12.0.0 published on 2026-07-08, 12.2.2 the
registry's latest today) reads no shrinkwrap at all: "As of npm v12, npm-shrinkwrap.json is no
longer read or written by npm. [...] When npm v12 encounters a dependency that ships an npm-shrinkwrap.json inside its tarball, the file is
ignored — publishers who need to ship a locked tree should use bundleDependencies instead." Every LTS
line of Node.js bundles npm 10 or 11 today (22.23.4: 10.9.9; 24.22.2: 11.19.1; 26.13.0: 11.20.0), and
27.0.0-alpha bundles npm 12.1.0. A shrinkwrap would have held for registry installs under npm 7 to 11,
and for nothing else: not for the offline install of a verified release asset that a regulated user
makes, and not past the next Node.js line.
What bundling does. With bundleDependencies naming the three direct dependencies, npm pack
carries their closure from the installed tree: 27 packages, 280 files, at the lock's versions. The
tarball installed from the file gave 27 of 27 ("added 1 package"); offline, with an empty cache and no
reachable registry, 27 of 27. Two packs on one machine are byte-identical. The tarball grows from 687,522
to 1,077,435 bytes; a fresh install today downloads 1,108,007 bytes in 28 tarballs (runward and the 27 at
the lock's versions), and afterwards one tarball of 1,077,435 bytes. npm audit still audits the bundled
packages (shouldAudit in arborist's audit-report.js excludes none), and npm audit signatures counts
the same 27 verified registry signatures and 19 attestations on the bundled install as on the unbundled
one.
The SBOM was a scan of the checkout. The v0.46.0 asset is CycloneDX 1.7 written by Syft 1.54.0 over
the repository: metadata.component is {"type": "file", "name": "."}, with no name or version of the
product; 117 components, of which 24 are files and 26 libraries outside the runtime tree (the actions of
the workflows, workflows inside development node_modules, floor-ts, the example); 34 carry a licence;
the versions are the lock's, which an install did not reproduce. It met the letter of "an SBOM is
attached" and would mislead a third-party risk tool reading it as the package's.
The job that signs ran the build. In build-and-attest.yml the job holding id-token: write, the
credential the provenance is signed with, also ran npm ci with install scripts enabled, npm test
(every test file and every development dependency they load) and npm pack. SLSA's Build track asks the
opposite: "Such secret material MUST NOT be accessible to the environment running the user-defined build
steps." (slsa.dev/spec/v1.2/build-requirements, "Provenance is Unforgeable", read 2026-11-05.) The same
was true in release.yml, where the job that signs the SBOM attestation ran npm ci, npm test and the
qualification kit against the installed tarball. runward asserts no SLSA level, and this record claims
none (test/unit/no-overclaim.test.js): it closes the gap between the property and the chain.
The release chain had been red at every release since 0.44.0. The publish job of 0.44.0, 0.44.1, 0.45.0
and 0.46.0 failed on its last step, npm error 404 Not Found - PUT https://registry.npmjs.org/runward-forge:
the package does not exist on npm, the one-time bootstrap of runward/runbook.md §3.1 being undone.
verify-release.yml runs only after a successful publish run, so it last verified a release on
2026-10-11 (0.43.1) and was skipped at each of the four. A version query cannot tell the two cases this
needs apart: npm view runward-forge and npm view chalk@99.0.0 both exit 1 with E404; the registry's
package document can (404 for a package that does not exist, 200 for one that does).
Three smaller gaps of the same review. Node.js 26 has been the active LTS line since 2026-10-28
(github.com/nodejs/Release) and the unit tests ran on 22 and 24 only. OpenSSF Scorecard's
Pinned-Dependencies read pip install compliance-trestle==4.2.0 (ci.yml) as unpinned: the package is
pinned, its 38 dependencies resolve afresh at every run. Eleven CodeQL alerts of severity high had been
open since 2026-09-29, none triaged.
Options considered
- Dependencies. A: a runtime-only
npm-shrinkwrap.json, derived from the lock at pack time. B:bundleDependencies. C: both. D: none; tell users to commit their own lock. - SBOM. a: Syft over the extracted tarball instead of the checkout. b:
npm sbomover an installation of the tarball. c: a deterministic generator over the tarball and the tag's lock, validated against the official CycloneDX schema. - Signing. i: one job builds, tests and packs without privilege; another, holding the signing credential, checks the digests of what it received and signs, and runs nothing else. ii: an external generator (a second builder identity). iii: keep the single job and say so.
- runward-forge. 1: let the publish step warn and succeed when the package does not exist on npm.
2: run
verify-release.ymlafter a failed publish too. 3: drop runward-forge from the release until the bootstrap.
Decision
B, c, i and 1 (decisions 1 to 4), and decisions 5 to 9.
- The tarball bundles its runtime dependencies.
package.jsondeclaresbundleDependencies, the names ofdependencies, andnpm packcarries their closure from the treenpm ciinstalled from the lock.test/unit/package-sbom.test.jspacks the tree and refuses a tarball whose bundle is not the lock's runtime closure at the lock's versions, so a Dependabot pull request that breaks it is red. - The release SBOM is the package's.
scripts/package-sbom.mjsreads the tarball and the tag'spackage-lock.jsonand writes CycloneDX 1.6:metadata.componentis the applicationrunward@X.Y.Zwith its purl, its licence and the SHA-256 and SHA-512 of the tarball; each bundled package is a component with its version, purl, declared licence, the SHA-512 of the registry tarball the lock names, its path and the propertycdx:npm:package:bundled; the dependency graph is resolved inside the bundle. It refuses (exit 1) a bundle that differs from the lock in any package, path, version or declared licence, or that cannot satisfy a dependency, and writes nothing then. The output is deterministic: the serial number is a UUID derived from the tarball's digest and the timestamp is the commit's. It uses Node.js built-ins only and no third-party action. The release'ssbomjob, which holdscontents: read, runs it on the builder's tarball; the cross-check job derives it again from its own checkout and refuses a difference;verify-release.ymlderives it a third time from the tag after publication (guide, Step 10). The asset keeps its name,runward-sbom.cdx.json, and its attestation (actions/attest, subject the tarball, predicate CycloneDX). The Syft job ofci.ymlstays, as what it is: a scan of the repository that shows drift, not the release's SBOM. The CycloneDX 1.6.1 schemas are vendored undertest/fixtures/and the unit test validates against them (ADR-0062's practice); the document this tree produces also passes cyclonedx-python-lib 11.5.0's strict validator. - No job that signs runs the build.
build-and-attest.ymlhas two jobs.buildholdscontents: readonly and runsnpm ci --ignore-scripts,npm test, both packs, the qualification kit and the digests of the three files.attestholdsid-token: writeandattestations: write, checks out nothing and runs no npm: it downloads the three files, refuses any whose SHA-256 differs from the digest the build job declared or whose name is not the expected one, signs the three provenances and hands them on under the artifact name the caller already reads.release.ymldoes the same for the SBOM attestation: the job that tests, runs the three byte-for-byte cross-checks (unchanged), runs the kit and derives the SBOM again isverify, withcontents: read;attestonly checks the tarball's and the SBOM's digests against the jobs that produced them and signs. Thedraftjob checks the same two digests before it uploads anything. A unit test holds the rule for every job ofrelease.ymlandbuild-and-attest.ymlthat holdsid-token: write: no checkout, nonpm ci,install,test,run,pack,execornpx, nonodeof a file of the tree, and no cache restored anywhere in the two files (a cache is the channel through which one run reaches another). The signing identity is unchanged: the certificate still namesbuild-and-attest.yml(orrelease.ymlfor the SBOM) at the tag, so no verification command changes. - A release whose runward-forge is not on npm stays green and says so. Before it publishes
runward-forge, the publish job asks the registry for the package document. 404: the package has never
been published, so npm cannot trust this workflow for it yet; the step writes a warning annotation and
a job-summary section naming runbook §3.1, and ends with exit 0, runward being already published and
the runward-forge tarball verified on the release. 200: unchanged, so a failed publish of a package
that exists stays red. Any other answer: exit 1.
verify-release.ymlthen runs after the release, as it did until 2026-10-11. - Node.js 26 joins the unit test matrix (22, 24, 26). Which checks are required stays the maintainer's setting; the pull request names the one to add.
- The OSCAL ingestion installs a hash-pinned closure.
test/oscal-ingest.requirements.innamescompliance-trestle==4.2.0;test/oscal-ingest.requirements.txtis its closure for CPython 3.12 on x86_64 Linux with every file's hash, generated byuv pip compile(the command is its header); CI installs it withpip install --require-hashes --only-binary=:all:, so no package outside it, no other file and no source build can enter the job. - CodeQL: the founded alerts are fixed, the others are argued, none is dismissed here. Fixed: a
hand-escaped
RegExpintest/unit/check-contract.test.js, and the host test offloor-ts' provider profile (a substring of the whole URL, now the parsed host name). Argued in the pull request: the six ReDoS alerts in the tests of the ReDoS screen (the patterns are its inputs, compiled and never matched) and the one insrc/lib/evidence.ts(the replaced string is one character long). The cells ofscripts/mutation-register.mjsare not changed here: GitHub's renderer shows the five rows first read as broken exactly as written (POST /markdown, modegfm, 2026-11-06), and the escaping tried at first, rendered over the whole register, showed 29 notes and 5 code cells wrong thatmainshows right (backslashes doubled inside code spans,\|shown as⧵|) for 6 notes put right. Dismissing an alert is the maintainer's act. - The kit runs against the tarball installed offline, and its runner handles a bundle. The release's
kit run installs the tarball with
--offlineand an empty cache, so a bundle that needs the registry fails the release instead of reaching npm. Run on a kit built from the bundled tarball, the runner (scripts/qualification-kit/run.mjs) linked the installation'snode_modules/into its work directory whole, wrote its dependency links through it into the installation, and stopped on EEXIST before any test. Linking each bundled package instead failed later: the citedexample-packed.test.jspacks the work directory, npm packs a linked bundled dependency under a path that leaves the package, andtarrefused it. The runner now copies into its work directory exactly the bundled files the digest comparison matched, and links nothing for a bundled installation (test/unit/qualification-kit.test.js, failing on the runner before the fix). - The reference JUnit report travels as an artifact (refines ADR-0112 decision 5). The
lint-and-reportsjob ofci.yml, which regeneratesreports/junit.xmlon macOS with the Node.js of.nvmrc, uploads the file it wrote as the artifactjunit-reference, even when the job fails. A person or a delegate commits that file, which carries<!-- generated: node vX.Y.Z darwin -->, without running the suite on their own machine. The job stays red while the committed report differs: the artifact corrects the tree, it never replaces the check.
Alternatives discarded
- A, the shrinkwrap. Measured above: honoured only for registry installs flagged by npmjs.org, under npm 7 to 11; ignored by any install from the file and by npm 12. C adds nothing to B: a bundled package is installed from the tarball, never resolved.
- D. It moves the lock to every user, and the tested tree still never reaches them.
- a. Closer, still a scan: a third-party action in the release path, a random serial number and a timestamp of the run, licences as its catalogers find them.
- b. CycloneDX 1.5, a random serial number and a timestamp, no component hash without a lockfile, and it needs an installation first.
- ii. A second builder identity every verification command would have to change to, while GitHub's own attestation actions already sign in a job runward chooses.
- iii. It leaves the signing credential within reach of every development dependency.
- 2. A run after a failed publish verifies whatever reached npm, or fails a second time on the same cause; decision 4 makes the expected failure a warning and leaves the others red.
- 3. The runward-forge tarball is already built, cross-checked and attested on every release; its absence from npm is a one-time gesture away (runbook §3.1).
Consequences
- The tarball is 57 % larger and a fresh install downloads about 3 % less, in one file instead of
- Its unpacked size grows by 1.2 MB.
- A user can no longer override a bundled package (
overrides) or deduplicate it with their own: a fix in a dependency reaches them through a runward release. Dependabot keeps opening those updates on runward's lock with its 7-day cooldown, security updates without it, and edits onlypackage.jsonand the lock, whichbundleDependencieschanges neither (Dependabot itself was not run). What such a pull request does to the bundle was measured by doing its edit by hand (npm install @inquirer/prompts@8.7.5 --package-lock-only, thennpm ci,npm pack): the bundle followed the lock (27 packages,@inquirer/prompts8.7.5), and the same tarball checked against the lock before the edit was refused, 16 differences, no SBOM written. - The qualification kit lists 280 more files and compares each of them on the user's installation;
an extra file under
node_modules/is still not reported, as before. - The release SBOM changes nature: a reader comparing it with an earlier release's sees a named product and its runtime tree, not a scan. The 0.46.0 and earlier assets stay as they were.
- The release run gains two jobs (the signing jobs); no verification command changes.
verify-release.ymlruns after every release again, and gains Step 10.- CI gains one Node.js 26 leg, not required until the maintainer makes it so.
test/oscal-ingest.requirements.txtis regenerated when trestle moves, by its header's command; GitHub's dependency graph now reads it, so an advisory on one of its 39 packages surfaces as an alert.
What this does not catch
- A build job that alters what it builds. The build job still runs every development dependency; one that changes the tarball before it is hashed gets it signed as what this run built. The byte-for-byte cross-check of a second job and the reproducible rebuild stay the controls, and both run the same dependencies: a change that reproduces itself in both packs passes both. Only a rebuild outside the chain, from a lock someone has read, sees it.
- Isolation between runs on GitHub-hosted runners is GitHub's; the chain's part is to restore no cache.
- Node.js, npm and the operating system are not in the bundle.
- The bundled bytes against the registry's. The SBOM hashes the registry tarballs the lock names;
npm civerified the installed files against those hashes in the build job, and the bundle is packed from what it installed. The SBOM does not re-hash the extracted files. - runward-forge itself: the warning does not publish it; the bootstrap stays the maintainer's.
What would settle it
- For decisions 1 and 2: on the first release that carries them,
npm install runward@X.Y.Zinto an empty project installs the 27 packages at the versions of the tag's lock, the guide's Step 10 re-derives the SBOM byte for byte, and a third-party CycloneDX reader (Dependency-Track,cyclonedx validate) accepts it. Against: an installation where a bundled package fails where the resolved one worked, or a reader that refuses the SBOM. - For decision 3: the first release's Steps 2, 2c and 3 pass unchanged, the certificate still names
build-and-attest.ymlat the tag, and the publish job's provenance checks pass. Against: a verification that needs to change to accept the new jobs. - For decision 4: while runward-forge is absent from npm, the publish run ends green with the warning
and
verify-release.ymlruns. Against: a publish run that ends green whilerunwardis not on npm, or one that is red for the absence of runward-forge.
Reevaluation trigger (mandatory, dated)
Reopen at the first release that carries this change (measure the install, the SBOM and the run against "What would settle it"), when npm changes how it installs bundled dependencies, when the CycloneDX readers of the ecosystem accept 1.7 widely, when runward-forge is bootstrapped on npm (decision 4's branch then goes quiet), or on 2027-02-05, whichever comes first.
Trigger set on: 2026-11-05 · Watched via: verify-release.yml (Step 10 and its conclusion), the
publish job's summary, test/unit/package-sbom.test.js and test/unit/release-signing-isolation.test.js.
References
package.json(bundleDependencies),scripts/package-sbom.mjs,scripts/tgz.mjs,scripts/qualification-kit/run.mjs(buildWorkDir),test/unit/package-sbom.test.js,test/unit/release-signing-isolation.test.js,test/fixtures/cyclonedx_bom_schema.v1.6.1.json,cyclonedx_spdx_schema.v1.6.1.json,cyclonedx_jsf_schema.v1.6.1.json,test/oscal-ingest.requirements.inand.txt..github/workflows/build-and-attest.yml,release.yml,verify-release.yml,ci.yml.docs/verifying-a-release.md(Steps 9 and 10),runward/runbook.md(§3 and §3.1).- npm 12.2.2,
docs/content/configuring-npm/package-lock-json.md; slsa.dev/spec/v1.2/build-requirements; github.com/nodejs/Releaseschedule.json; CycloneDX specification 1.6.1; cmark-gfmextensions/ext_scanners.re. - ADR-0031, ADR-0048, ADR-0049, ADR-0062, ADR-0085, ADR-0088 decisions 4 and 7, ADR-0094, ADR-0110, ADR-0112.