Aller au contenu
runward
runward
Produit
Pour qui
Confiance
Documentation
Nouveautés
Essayer sur un exemple Échanger avec nous

Docs · Décisions · ADR-0116

ADR-0116

ADR-0116-what-a-user-installs-is-what-the-release-tested-the-tarball-bundles-its-dependencies-the-sbom-is-the-packages-and-no-signing-job-runs-the-build.md

ADR-0116 — What a user installs is what the release tested: the tarball bundles its runtime dependencies, the SBOM describes the package, and no job that signs runs the build

Date: 2026-11-05 Status: accepted (2026-11-05) — the release tarball carries its runtime dependencies at the versions of the tag's lockfile (bundleDependencies), the release SBOM is the package's own (CycloneDX 1.6, derived from the tarball and the lockfile, re-derivable by anyone), the jobs that hold id-token: write in the release chain check out nothing and run no code of the tree or its dependencies, a release whose runward-forge is not yet on npm publishes runward, warns and stays green, Node.js 26 joins the unit test matrix, the OSCAL ingestion installs from a hash-pinned closure, and the CI job that writes the reference JUnit report keeps it as an artifact Deciders: delegate agent, class D of runward/delegation.md, on the maintainer's instruction of 2026-11-05. The change touches .github/** and package.json, pre-merge paths of the charter: the pull request waits for the maintainer before merge (ADR-0088 decision 7). Under ADR-0088 decision 4 the maintainer is the single accountable person, and the agent's writing is disclosed throughput, not independent review Method: measured on 2026-11-05 on a fresh clone of stranxik/runward at main@8ffdf7c (0.46.0), macOS, Node.js 24.22.1, npm 11.19.1, every install into an empty project with its own npm cache and an empty user configuration: the tarball of main installed as it is; the same tarball with a runtime-only npm-shrinkwrap.json (derived from package-lock.json, 28 entries, 15,058 bytes), installed from the file and from a one-package local registry that serves it with _hasShrinkwrap set to true, then to false, every other request redirected to registry.npmjs.org; the tarball packed with bundleDependencies, installed from the file online, then offline with an empty cache and an unreachable registry; npm audit signatures on the bundled and unbundled installs; on the committed change, the guide's Step 10 replayed on a tarball packed from it (cmp equal) and that tarball installed from the file, offline and from a local registry (27 of 27 each). Read: the v0.46.0 release assets (runward-sbom.cdx.json), .github/workflows/build-and-attest.yml, release.yml, verify-release.yml, ci.yml, the npm 11.19.1 and 12.2.2 sources (@npmcli/arborist, npm-packlist, lib/utils/verify-signatures.js) and npm 12.2.2's docs/content/configuring-npm/package-lock-json.md, the registry's package documents of runward, sfdx-cli, @salesforce/cli and firebase-tools, nodejs.org/dist/index.json and github.com/nodejs/Release schedule.json, slsa.dev/spec/v1.2 (Build track basics and requirements), the GitHub API (release and verify-release runs, the failed step's log of run 40684642168, the open code-scanning alerts, all read with GET), and cmark-gfm's table scanner (extensions/ext_scanners.re, extensions/table.c) Relates to: ADR-0031 (the supply-chain evidence a regulated buyer reads), ADR-0048, ADR-0049 (the isolated builder, refined here), ADR-0062 (an emitted document is validated against the official schema), ADR-0085, ADR-0094 (runward-forge), ADR-0110, ADR-0112 (the reference environment, refined by decision 9)

Context

What a user ran was not what the release tested. The tarball declared three dependencies by range and carried no lock. Installed into an empty project, the 0.46.0 tarball resolved 16 of its 27 runtime packages to other versions than the tag's package-lock.json (every @inquirer/* package, by a patch release each); the suite, the self-gate and the SBOM had all used the lock's versions. For a tool whose promise is the same verdict on the same tree, the code that runs at an adopter's was neither the tested code nor the code the SBOM listed.

What a shrinkwrap would and would not have fixed. A runtime-only npm-shrinkwrap.json in the same tarball changed nothing for an install from the file: 11 of 27 again. npm reads a dependency's shrinkwrap only when the registry's metadata marks the version _hasShrinkwrap: true (node.hasShrinkwrap in @npmcli/arborist), which an install from a file never has. Served by a local registry with the flag true, the same tarball installed 27 of 27; with the flag false, 11 of 27. npmjs.org sets the flag from the tarball (it is true on sfdx-cli 7.209.6 and on every @salesforce/cli version published with npm 10 and 11, false on runward 0.46.0, and the npm 11 client writes no such field: an inference, since the registry's code is not public). And npm 12 (12.0.0 published on 2026-07-08, 12.2.2 the registry's latest today) reads no shrinkwrap at all: "As of npm v12, npm-shrinkwrap.json is no longer read or written by npm. [...] When npm v12 encounters a dependency that ships an npm-shrinkwrap.json inside its tarball, the file is ignored — publishers who need to ship a locked tree should use bundleDependencies instead." Every LTS line of Node.js bundles npm 10 or 11 today (22.23.4: 10.9.9; 24.22.2: 11.19.1; 26.13.0: 11.20.0), and 27.0.0-alpha bundles npm 12.1.0. A shrinkwrap would have held for registry installs under npm 7 to 11, and for nothing else: not for the offline install of a verified release asset that a regulated user makes, and not past the next Node.js line.

What bundling does. With bundleDependencies naming the three direct dependencies, npm pack carries their closure from the installed tree: 27 packages, 280 files, at the lock's versions. The tarball installed from the file gave 27 of 27 ("added 1 package"); offline, with an empty cache and no reachable registry, 27 of 27. Two packs on one machine are byte-identical. The tarball grows from 687,522 to 1,077,435 bytes; a fresh install today downloads 1,108,007 bytes in 28 tarballs (runward and the 27 at the lock's versions), and afterwards one tarball of 1,077,435 bytes. npm audit still audits the bundled packages (shouldAudit in arborist's audit-report.js excludes none), and npm audit signatures counts the same 27 verified registry signatures and 19 attestations on the bundled install as on the unbundled one.

The SBOM was a scan of the checkout. The v0.46.0 asset is CycloneDX 1.7 written by Syft 1.54.0 over the repository: metadata.component is {"type": "file", "name": "."}, with no name or version of the product; 117 components, of which 24 are files and 26 libraries outside the runtime tree (the actions of the workflows, workflows inside development node_modules, floor-ts, the example); 34 carry a licence; the versions are the lock's, which an install did not reproduce. It met the letter of "an SBOM is attached" and would mislead a third-party risk tool reading it as the package's.

The job that signs ran the build. In build-and-attest.yml the job holding id-token: write, the credential the provenance is signed with, also ran npm ci with install scripts enabled, npm test (every test file and every development dependency they load) and npm pack. SLSA's Build track asks the opposite: "Such secret material MUST NOT be accessible to the environment running the user-defined build steps." (slsa.dev/spec/v1.2/build-requirements, "Provenance is Unforgeable", read 2026-11-05.) The same was true in release.yml, where the job that signs the SBOM attestation ran npm ci, npm test and the qualification kit against the installed tarball. runward asserts no SLSA level, and this record claims none (test/unit/no-overclaim.test.js): it closes the gap between the property and the chain.

The release chain had been red at every release since 0.44.0. The publish job of 0.44.0, 0.44.1, 0.45.0 and 0.46.0 failed on its last step, npm error 404 Not Found - PUT https://registry.npmjs.org/runward-forge: the package does not exist on npm, the one-time bootstrap of runward/runbook.md §3.1 being undone. verify-release.yml runs only after a successful publish run, so it last verified a release on 2026-10-11 (0.43.1) and was skipped at each of the four. A version query cannot tell the two cases this needs apart: npm view runward-forge and npm view chalk@99.0.0 both exit 1 with E404; the registry's package document can (404 for a package that does not exist, 200 for one that does).

Three smaller gaps of the same review. Node.js 26 has been the active LTS line since 2026-10-28 (github.com/nodejs/Release) and the unit tests ran on 22 and 24 only. OpenSSF Scorecard's Pinned-Dependencies read pip install compliance-trestle==4.2.0 (ci.yml) as unpinned: the package is pinned, its 38 dependencies resolve afresh at every run. Eleven CodeQL alerts of severity high had been open since 2026-09-29, none triaged.

Options considered

  • Dependencies. A: a runtime-only npm-shrinkwrap.json, derived from the lock at pack time. B: bundleDependencies. C: both. D: none; tell users to commit their own lock.
  • SBOM. a: Syft over the extracted tarball instead of the checkout. b: npm sbom over an installation of the tarball. c: a deterministic generator over the tarball and the tag's lock, validated against the official CycloneDX schema.
  • Signing. i: one job builds, tests and packs without privilege; another, holding the signing credential, checks the digests of what it received and signs, and runs nothing else. ii: an external generator (a second builder identity). iii: keep the single job and say so.
  • runward-forge. 1: let the publish step warn and succeed when the package does not exist on npm. 2: run verify-release.yml after a failed publish too. 3: drop runward-forge from the release until the bootstrap.

Decision

B, c, i and 1 (decisions 1 to 4), and decisions 5 to 9.

  1. The tarball bundles its runtime dependencies. package.json declares bundleDependencies, the names of dependencies, and npm pack carries their closure from the tree npm ci installed from the lock. test/unit/package-sbom.test.js packs the tree and refuses a tarball whose bundle is not the lock's runtime closure at the lock's versions, so a Dependabot pull request that breaks it is red.
  2. The release SBOM is the package's. scripts/package-sbom.mjs reads the tarball and the tag's package-lock.json and writes CycloneDX 1.6: metadata.component is the application runward@X.Y.Z with its purl, its licence and the SHA-256 and SHA-512 of the tarball; each bundled package is a component with its version, purl, declared licence, the SHA-512 of the registry tarball the lock names, its path and the property cdx:npm:package:bundled; the dependency graph is resolved inside the bundle. It refuses (exit 1) a bundle that differs from the lock in any package, path, version or declared licence, or that cannot satisfy a dependency, and writes nothing then. The output is deterministic: the serial number is a UUID derived from the tarball's digest and the timestamp is the commit's. It uses Node.js built-ins only and no third-party action. The release's sbom job, which holds contents: read, runs it on the builder's tarball; the cross-check job derives it again from its own checkout and refuses a difference; verify-release.yml derives it a third time from the tag after publication (guide, Step 10). The asset keeps its name, runward-sbom.cdx.json, and its attestation (actions/attest, subject the tarball, predicate CycloneDX). The Syft job of ci.yml stays, as what it is: a scan of the repository that shows drift, not the release's SBOM. The CycloneDX 1.6.1 schemas are vendored under test/fixtures/ and the unit test validates against them (ADR-0062's practice); the document this tree produces also passes cyclonedx-python-lib 11.5.0's strict validator.
  3. No job that signs runs the build. build-and-attest.yml has two jobs. build holds contents: read only and runs npm ci --ignore-scripts, npm test, both packs, the qualification kit and the digests of the three files. attest holds id-token: write and attestations: write, checks out nothing and runs no npm: it downloads the three files, refuses any whose SHA-256 differs from the digest the build job declared or whose name is not the expected one, signs the three provenances and hands them on under the artifact name the caller already reads. release.yml does the same for the SBOM attestation: the job that tests, runs the three byte-for-byte cross-checks (unchanged), runs the kit and derives the SBOM again is verify, with contents: read; attest only checks the tarball's and the SBOM's digests against the jobs that produced them and signs. The draft job checks the same two digests before it uploads anything. A unit test holds the rule for every job of release.yml and build-and-attest.yml that holds id-token: write: no checkout, no npm ci, install, test, run, pack, exec or npx, no node of a file of the tree, and no cache restored anywhere in the two files (a cache is the channel through which one run reaches another). The signing identity is unchanged: the certificate still names build-and-attest.yml (or release.yml for the SBOM) at the tag, so no verification command changes.
  4. A release whose runward-forge is not on npm stays green and says so. Before it publishes runward-forge, the publish job asks the registry for the package document. 404: the package has never been published, so npm cannot trust this workflow for it yet; the step writes a warning annotation and a job-summary section naming runbook §3.1, and ends with exit 0, runward being already published and the runward-forge tarball verified on the release. 200: unchanged, so a failed publish of a package that exists stays red. Any other answer: exit 1. verify-release.yml then runs after the release, as it did until 2026-10-11.
  5. Node.js 26 joins the unit test matrix (22, 24, 26). Which checks are required stays the maintainer's setting; the pull request names the one to add.
  6. The OSCAL ingestion installs a hash-pinned closure. test/oscal-ingest.requirements.in names compliance-trestle==4.2.0; test/oscal-ingest.requirements.txt is its closure for CPython 3.12 on x86_64 Linux with every file's hash, generated by uv pip compile (the command is its header); CI installs it with pip install --require-hashes --only-binary=:all:, so no package outside it, no other file and no source build can enter the job.
  7. CodeQL: the founded alerts are fixed, the others are argued, none is dismissed here. Fixed: a hand-escaped RegExp in test/unit/check-contract.test.js, and the host test of floor-ts' provider profile (a substring of the whole URL, now the parsed host name). Argued in the pull request: the six ReDoS alerts in the tests of the ReDoS screen (the patterns are its inputs, compiled and never matched) and the one in src/lib/evidence.ts (the replaced string is one character long). The cells of scripts/mutation-register.mjs are not changed here: GitHub's renderer shows the five rows first read as broken exactly as written (POST /markdown, mode gfm, 2026-11-06), and the escaping tried at first, rendered over the whole register, showed 29 notes and 5 code cells wrong that main shows right (backslashes doubled inside code spans, \| shown as ⧵|) for 6 notes put right. Dismissing an alert is the maintainer's act.
  8. The kit runs against the tarball installed offline, and its runner handles a bundle. The release's kit run installs the tarball with --offline and an empty cache, so a bundle that needs the registry fails the release instead of reaching npm. Run on a kit built from the bundled tarball, the runner (scripts/qualification-kit/run.mjs) linked the installation's node_modules/ into its work directory whole, wrote its dependency links through it into the installation, and stopped on EEXIST before any test. Linking each bundled package instead failed later: the cited example-packed.test.js packs the work directory, npm packs a linked bundled dependency under a path that leaves the package, and tar refused it. The runner now copies into its work directory exactly the bundled files the digest comparison matched, and links nothing for a bundled installation (test/unit/qualification-kit.test.js, failing on the runner before the fix).
  9. The reference JUnit report travels as an artifact (refines ADR-0112 decision 5). The lint-and-reports job of ci.yml, which regenerates reports/junit.xml on macOS with the Node.js of .nvmrc, uploads the file it wrote as the artifact junit-reference, even when the job fails. A person or a delegate commits that file, which carries <!-- generated: node vX.Y.Z darwin -->, without running the suite on their own machine. The job stays red while the committed report differs: the artifact corrects the tree, it never replaces the check.

Alternatives discarded

  • A, the shrinkwrap. Measured above: honoured only for registry installs flagged by npmjs.org, under npm 7 to 11; ignored by any install from the file and by npm 12. C adds nothing to B: a bundled package is installed from the tarball, never resolved.
  • D. It moves the lock to every user, and the tested tree still never reaches them.
  • a. Closer, still a scan: a third-party action in the release path, a random serial number and a timestamp of the run, licences as its catalogers find them.
  • b. CycloneDX 1.5, a random serial number and a timestamp, no component hash without a lockfile, and it needs an installation first.
  • ii. A second builder identity every verification command would have to change to, while GitHub's own attestation actions already sign in a job runward chooses.
  • iii. It leaves the signing credential within reach of every development dependency.
  • 2. A run after a failed publish verifies whatever reached npm, or fails a second time on the same cause; decision 4 makes the expected failure a warning and leaves the others red.
  • 3. The runward-forge tarball is already built, cross-checked and attested on every release; its absence from npm is a one-time gesture away (runbook §3.1).

Consequences

  • The tarball is 57 % larger and a fresh install downloads about 3 % less, in one file instead of
    1. Its unpacked size grows by 1.2 MB.
  • A user can no longer override a bundled package (overrides) or deduplicate it with their own: a fix in a dependency reaches them through a runward release. Dependabot keeps opening those updates on runward's lock with its 7-day cooldown, security updates without it, and edits only package.json and the lock, which bundleDependencies changes neither (Dependabot itself was not run). What such a pull request does to the bundle was measured by doing its edit by hand (npm install @inquirer/prompts@8.7.5 --package-lock-only, then npm ci, npm pack): the bundle followed the lock (27 packages, @inquirer/prompts 8.7.5), and the same tarball checked against the lock before the edit was refused, 16 differences, no SBOM written.
  • The qualification kit lists 280 more files and compares each of them on the user's installation; an extra file under node_modules/ is still not reported, as before.
  • The release SBOM changes nature: a reader comparing it with an earlier release's sees a named product and its runtime tree, not a scan. The 0.46.0 and earlier assets stay as they were.
  • The release run gains two jobs (the signing jobs); no verification command changes.
  • verify-release.yml runs after every release again, and gains Step 10.
  • CI gains one Node.js 26 leg, not required until the maintainer makes it so.
  • test/oscal-ingest.requirements.txt is regenerated when trestle moves, by its header's command; GitHub's dependency graph now reads it, so an advisory on one of its 39 packages surfaces as an alert.

What this does not catch

  • A build job that alters what it builds. The build job still runs every development dependency; one that changes the tarball before it is hashed gets it signed as what this run built. The byte-for-byte cross-check of a second job and the reproducible rebuild stay the controls, and both run the same dependencies: a change that reproduces itself in both packs passes both. Only a rebuild outside the chain, from a lock someone has read, sees it.
  • Isolation between runs on GitHub-hosted runners is GitHub's; the chain's part is to restore no cache.
  • Node.js, npm and the operating system are not in the bundle.
  • The bundled bytes against the registry's. The SBOM hashes the registry tarballs the lock names; npm ci verified the installed files against those hashes in the build job, and the bundle is packed from what it installed. The SBOM does not re-hash the extracted files.
  • runward-forge itself: the warning does not publish it; the bootstrap stays the maintainer's.

What would settle it

  • For decisions 1 and 2: on the first release that carries them, npm install runward@X.Y.Z into an empty project installs the 27 packages at the versions of the tag's lock, the guide's Step 10 re-derives the SBOM byte for byte, and a third-party CycloneDX reader (Dependency-Track, cyclonedx validate) accepts it. Against: an installation where a bundled package fails where the resolved one worked, or a reader that refuses the SBOM.
  • For decision 3: the first release's Steps 2, 2c and 3 pass unchanged, the certificate still names build-and-attest.yml at the tag, and the publish job's provenance checks pass. Against: a verification that needs to change to accept the new jobs.
  • For decision 4: while runward-forge is absent from npm, the publish run ends green with the warning and verify-release.yml runs. Against: a publish run that ends green while runward is not on npm, or one that is red for the absence of runward-forge.

Reevaluation trigger (mandatory, dated)

Reopen at the first release that carries this change (measure the install, the SBOM and the run against "What would settle it"), when npm changes how it installs bundled dependencies, when the CycloneDX readers of the ecosystem accept 1.7 widely, when runward-forge is bootstrapped on npm (decision 4's branch then goes quiet), or on 2027-02-05, whichever comes first.

Trigger set on: 2026-11-05 · Watched via: verify-release.yml (Step 10 and its conclusion), the publish job's summary, test/unit/package-sbom.test.js and test/unit/release-signing-isolation.test.js.

References

  • package.json (bundleDependencies), scripts/package-sbom.mjs, scripts/tgz.mjs, scripts/qualification-kit/run.mjs (buildWorkDir), test/unit/package-sbom.test.js, test/unit/release-signing-isolation.test.js, test/fixtures/cyclonedx_bom_schema.v1.6.1.json, cyclonedx_spdx_schema.v1.6.1.json, cyclonedx_jsf_schema.v1.6.1.json, test/oscal-ingest.requirements.in and .txt.
  • .github/workflows/build-and-attest.yml, release.yml, verify-release.yml, ci.yml.
  • docs/verifying-a-release.md (Steps 9 and 10), runward/runbook.md (§3 and §3.1).
  • npm 12.2.2, docs/content/configuring-npm/package-lock-json.md; slsa.dev/spec/v1.2/build-requirements; github.com/nodejs/Release schedule.json; CycloneDX specification 1.6.1; cmark-gfm extensions/ext_scanners.re.
  • ADR-0031, ADR-0048, ADR-0049, ADR-0062, ADR-0085, ADR-0088 decisions 4 and 7, ADR-0094, ADR-0110, ADR-0112.
← Tous les ADR

Échanger avec nous

Dites-nous qui vous êtes : nous revenons vers vous.

Vos coordonnées sont enregistrées par Another Framework, éditeur de runward, pour vous recontacter. Elles ne sont ni vendues ni partagées, et sont effacées trois ans après notre dernier échange. Pour les consulter ou les effacer : contact@runward.dev.