Model-bundle manifest hashing

Status

Accepted — principle adopted, construction revised — 23 August 2026 (owner review).

The principle — a canonical manifest, hashed — is accepted. Four revisions attach, and all four are in standards/ver/1.1-draft/VER-1.1-draft.md §4 and the draft schema:

1. weights_sha256 is NOT redefined. It keeps its 1.0 reading (annex E29) forever. The 1.1 digest is a new field, model.bundle.manifest_sha256.
2. The construction is intrinsic to a versioned identifier, bundle_digest_alg: "ver-bundle-manifest/1", and the decorative canonicalization member is dropped.
3. The path grammar is hardened — the original pattern accepted ..\evil, C:\evil and a\..\evil.
4. model.revision is required for a remotely resolvable checkpoint, and typed model.artifacts[] carries the non-checkpoint files that affect a space's output.

Nothing in VER 1.0.x changes either way.

Context

§6.1 makes weight pinning mandatory:

model — family, architecture, checkpoint identifier, and weights_sha256. Weight pinning is mandatory.

§12 states what the pin is for:

weights_sha256 pinning plus space immutability prevents silent weight swaps and drift.

The schema constrains weights_sha256 to ^[a-f0-9]{64}$ and stops there. It never says what is hashed (Appendix D #5). For a single-file checkpoint the obvious reading is the file's SHA-256, but even that is not stated, and the obvious reading does not survive contact with reality: modern checkpoints are sharded (model-00001-of-00003.safetensors plus an index), and many are published in two formats simultaneously.

The audit found this failing in every way it can fail. In the reference implementation:

• The filename is folded into the digest preimage, so even a single-file checkpoint's weights_sha256 is not the file's SHA-256 — verified numerically (SC-01).
• The file set depends on the local cache. A machine that happens to hold both model.safetensors and pytorch_model.bin digests a different byte set from one holding only the first, so two conformant producers emit different weights_sha256 for the same space_id (SC-02).
• The preimage concatenates name and content with no length framing, the exact ambiguity class §4's rationale eliminates for pixel_hash: sha256(b"model.safetensors" + b"AB") == sha256(b"model.safetensorsA" + b"B") (SC-03).
• No revision is pinned, so a fixed space_id can denote different weights over time — a direct §6.2 immutability violation (SC-05).

A mandatory field that no consumer can reproduce is decorative. Worse, it is actively misleading: it looks like the anti-drift guarantee §12 promises, and it is not one.

Options

(a) Single canonical file only. weights_sha256 is the SHA-256 of exactly one designated weights file (safetensors preferred). Trivially reproducible. Cannot express a sharded checkpoint at all — which is most large models — so producers would pin the index file, or a shard, or invent something.

(b) Ordered concatenation of file bytes. Sort files by path, concatenate contents, hash. Simple, no manifest format needed. Requires streaming every byte to verify, gives no per-file verifiability, and reintroduces framing ambiguity unless lengths are included — at which point it is a worse manifest.

(c) A canonical bundle manifest, hashed. Build a manifest naming every file with its length and digest, canonicalize it deterministically, hash that. The digest is over metadata rather than bytes, so verification is cheap and per-file; the cost is a defined manifest format and a defined canonicalization.

Decision

Option (c), with the four revisions in the Status.

manifest_sha256 := SHA-256( RFC 8785 JCS canonical form of
                            { "files": [ { "path": …,
                                           "byte_length": …,
                                           "sha256": … }, … ] } )

with these rules:

1. weights_sha256 is not touched. Retrofitting a preimage onto a published field silently changes the meaning of every value already in circulation — a major change wearing a minor's clothing, and one no consumer could detect. weights_sha256 keeps annex E29's reading in perpetuity: SHA-256, lowercase hex, over an artifact the Producer chooses and SHOULD document, comparable only within a producer. The reproducible cross-producer pin is the new field, model.bundle.manifest_sha256, and 1.1 §4.2 says which is which so that no consumer compares the wrong one.
2. The construction is named by bundle_digest_alg, `const "ver-bundle-manifest/1"`, and that identifier is the definition above. A future construction — a different canonicalization, a wider path repertoire, a different hash — is a new identifier, never a new reading of this one. The separate canonicalization: "rfc8785-jcs" member is dropped: a decorative member every producer sets to one constant tells a verifier nothing, and two self-descriptions of one construction can disagree.
3. Sort by path, byte-wise ascending over the UTF-8 encoding, and store the entries in that order. A verifier checks the order it received rather than sorting first, so a tampered order is detected instead of repaired. Paths MUST be unique within a manifest; duplicate or aliasing entries (two entries naming one file) are non-conformant.
4. The path grammar is hardened. The original draft's pattern was written to exclude ., .. and a leading /, and it did — while accepting ..\evil, C:\evil and a\..\evil, because it constrained the shape of a segment and never its alphabet. 1.1 encodes: / as the only separator; non-empty segments, never . or ..; no leading /; and every character drawn from the printable portable set [A-Za-z0-9._-], which rejects backslash, colon, NUL and control characters by construction. A complement-class not keyword accompanies the anchored pattern so that a validator whose $ matches before a trailing newline cannot admit one. A wider repertoire (Unicode NFC, case-sensitive comparison) may be defined later — by a new bundle_digest_alg, not by relaxing this one. **Verification MUST resolve every path beneath a verified root and MUST NOT follow symbolic links out of it**; the grammar makes a traversal payload unrepresentable, and this rule covers the symlink case the grammar cannot reach.
5. byte_length is the file's exact length in octets. Present for framing: with lengths in the manifest, SC-03's concatenation ambiguity cannot arise.
6. sha256 is the file's own SHA-256, lowercase hex, over its bytes and nothing else. In particular the filename is not folded in — that is SC-01 exactly.
7. RFC 8785 (JCS) canonicalizes the manifest, the same canonicalization §9's record signature uses. One canonicalization rule for the standard, not two.
8. **Membership is the space publisher's decision, and the digest proves the integrity of the LIST, not its completeness.** This is stated in the normative text rather than left to the Consequences: no digest can attest to a file nobody listed, so a manifest with correct per-file digests and a missing shard is self-consistent and incomplete. The bundle lists the checkpoint's files, not the cache's — exactly one weight format; prefer safetensors; where sharded, the index plus its shards; ignore other formats even when present locally (SC-02). It is also the space's, not the whole checkpoint's: 1.1 §4.2 defines “affects the model's output” as the forward pass the space declares, so an image-only space over a joint image-text checkpoint lists the image tower's files and not the tokenizer, and two spaces over one checkpoint that declare different forward passes have different manifest_sha256 values by construction.
9. The manifest is carried inline, so a consumer can verify without re-deriving the file list. There is no by-reference form: $defs/bundleManifest permits nothing but the algorithm, the digest and the files. A by-reference bundle ({uri, sha256}) was left out deliberately — the manifest is small, a reference makes verification depend on a fetch, and the digest it would carry is already manifest_sha256 itself.
10. Single-file checkpoints are not special-cased. A one-entry manifest. One rule, no branch, no ambiguity about which rule applies.
11. model.revision is REQUIRED for a remotely resolvable checkpoint. Revision pinning was left out of the original ADR as a separate defect; that was wrong, because a manifest that fixes what was hashed while the checkpoint identifier still names a mutable branch leaves SC-05 exactly where it was. 1.1 adds model.checkpoint_uri and encodes checkpoint_uri ⇒ revision. The unconditional half — state the URI when the checkpoint is remotely resolvable — is profile material, because no schema can tell whether a bare checkpoint string names a remote repository.
12. model.artifacts[] carries the non-checkpoint files. A typed array of {role, path?, uri?, sha256, byte_length} with role ∈ tokenizer, pca_basis, codebook, processor, normalizer, other; at least one of path/uri, and sha256/byte_length always. These affect a space's output without being part of the checkpoint, and they are part of the space's identity under §6.2. Putting them in bundle would make the file list depend on how a Producer packages its pipeline; leaving them to ad-hoc preprocessing members (pca_basis_sha256 and its kind) leaves each deployment to invent a spelling. Where an artifact is also named by an E13 preprocessing member — the text half's tokenizer_sha256 — the two values MUST be equal; preprocessing keeps the member because E13 defines the full descriptor for 1.0.x and VER405 grades it.

For 1.0.x, annex E29 records that VER 1.0.x defines no preimage for weights_sha256 beyond “SHA-256, lowercase hex”, that a multi-file checkpoint therefore has no interoperable construction, and that 1.0.x values are comparable only within a producer; it asks Producers to document which artifact they hashed. (Annex E22 covers digest encodings and the registered perceptual algorithms — it is not the item that carries this reading, and citing it for this was a mis-citation in an earlier revision of this ADR.) Retrofitting this construction into 1.0 would silently change the meaning of values already published, which is a major change — and under rule 0 it is not retrofitted into 1.1 either. E29's reading is weights_sha256's reading in every release of VER, and the new construction lives in a new field.

Consequences

Good.

• The §6.1 pin becomes reproducible by a third party — the property that makes §12's supply-chain claim true rather than aspirational — **without changing what any published value meant**. That is the whole benefit of adding a field instead of redefining one: no 1.0 Record's weights_sha256 acquires a new reading, and no consumer's cached comparison silently changes answer.
• Verification is cheap and incremental: check the manifest's own digest, then spot-check individual files. Option (b) requires streaming everything.
• Per-file digests localise a mismatch. "Shard 2 of 3 differs" is actionable; "the bundle differs" is not.
• The path rule makes the layout part of the identity, which is correct: a bundle rearranged is a bundle changed, and the audit's cache-dependence bug is precisely a layout leak. The hardened grammar additionally means a manifest path cannot be a traversal payload for a consumer that joins it onto a cache root — the class of defect MOD-01 and WIRE-06 are instances of.
• bundle_digest_alg gives the construction a name, so revising it later is an addition rather than an argument about which reading a 64-hex string had.
• Reuses RFC 8785, already required for record signatures (annex E17). No new canonicalization to specify, implement, or get wrong.

Bad.

• Two digests now live in one model object, and they mean different things: weights_sha256 is producer-scoped and manifest_sha256 is not. A consumer that compares the wrong one gets a wrong answer quietly. 1.1 §4.2 states which is which, and COMPATIBILITY.md repeats it, because a documentation-only mitigation for a footgun is what this is.
• The digest is over metadata, not over weight bytes. A manifest with correct per-file digests but a wrong file list — an omitted shard — produces a self-consistent manifest describing an incomplete bundle. Membership is the publisher's decision and nothing in the digest enforces completeness; rule 7 now says so normatively rather than leaving it here.
• The portable path charset [A-Za-z0-9._-] excludes real filenames: spaces, non-ASCII characters, and anything a non-English checkpoint author might use. That is a deliberate narrow start — a permissive path rule is where this class of digest goes wrong across platforms — but a producer whose checkpoint contains such a file cannot express its bundle under ver-bundle-manifest/1 and must wait for a /2.
• Producers must retain the file list. A pipeline that loads a model through a framework's cache API may not know which files it opened, which is exactly how the reference implementation ended up hashing whatever it found (SC-02).
• artifacts[] overlaps preprocessing.tokenizer_sha256 by design, so one value can be stated twice and the two can disagree. The MUST-be-equal rule is a two-member comparison no schema can make; it is profile material.

Neutral.

• The wire profile carries no weights_sha256 at all (WIRE-08), so nothing on the wire changes.
• If ratification reverses this ADR, bundle, artifacts[], checkpoint_uri and manifest_sha256 leave the 1.1 schema and weights_sha256 is exactly where E29 left it. No published artifact changes; only the draft does.