§8 Richer conformance evidence

8.1 The problem

1.0 §6.2 defines conformance to a space as a mean cosine similarity against published reference vectors. An implementation with 999 vectors at 1.0 and one at 0.0 has a mean of 0.999 and passes a 0.999 tolerance — while being catastrophically wrong on one input. Worse, the reference block is optional, so most descriptors in circulation declare no tolerance at all and 1.0 §6.2's test cannot be run (E13).

8.2 Normative text (DRAFT)

spaceDescriptor.reference gains four members:

• set_manifest_sha256 — SHA-256 of the RFC 8785 JCS serialization of the reference-set manifest, binding the declared tolerances to an exact set of reference images and reference vectors. It is REQUIRED: a reference block is a conformance claim, and a claim bound only to a URI is a claim about whatever that URI serves today (§15.3).
• tolerance_cosine_p5 — the 5th-percentile per-item cosine floor.
• tolerance_cosine_worst — the worst single-item cosine floor.
• tolerance_hamming_max — the native binary-space tolerance (§11.2).

An implementation conforms to a space iff every declared tolerance is met: the mean meets tolerance_cosine, and where declared, the 5th percentile meets tolerance_cosine_p5 and the worst item meets tolerance_cosine_worst. Mean-only conformance is deprecated in 1.1: descriptors published under 1.1 SHOULD declare all three.

Reference-set manifest. The manifest names, for each reference item, the image's content_hash, its expected Canonical Buffer digest (pixel_hash), and the digest of the expected vector bytes. Publishing the manifest — not merely a URI — is what makes a tolerance a testable claim.

Floors, by dtype. For fp32, fp16 and bf16 spaces, tolerance_cosine is REQUIRED wherever a reference block is present and MUST be ≥ 0.99 (schema-encoded); it SHOULD be ≥ 0.999. An int8 space MUST declare tolerance_cosine, MAY declare it below the float floor, and MUST justify the value in the published descriptor. A binary space declares tolerance_hamming_max instead — see §11.2.

A descriptor without reference makes no conformance claim (E13). In 1.1 this is stated in the specification rather than only in the profile: a Producer MUST NOT advertise vectors from a reference-less space as interoperable, and a Consumer MUST NOT compare them across producers.

The 0.99 floor was profile-only guidance in 1.0.x (annex note N5). In 1.1 it is schema-encoded, because a conformance bar that anything clears is not a bar (SCH-20).

8.3 The decoder reference corpus

E12 defines a conformant decoder in terms of a published implementation profile and reference corpus, and notes that publishing the first one is a 1.1 deliverable. The first one now exists: standards/decoder-corpus/1/, profile verd-pillow/1, re-derived in CI on every commit (RELEASE.md row 9). The requirement it discharges is unchanged — the corpus MUST exist before 1.1 ratifies. What it does not discharge is E12's other half: PROVISIONAL lifts only between implementations evaluated against a published profile, and exactly one has been evaluated — the one that generated the corpus. Until a second, independently built decoder reproduces the corpus (rows 14 and the §3.4 bar), cross-producer pixel_hash claims remain claims about one implementation's determinism, and this draft says so rather than implying a guarantee it cannot make.