embedding.dim and dtype become REQUIRED for 1.1 Records

Status

Accepted as written — 23 August 2026 (owner review). Drafted at standards/ver/1.1-draft/VER-1.1-draft.md §12. Applies only to Records declaring ver_version: "1.1"; VER 1.0.x Records are unaffected in perpetuity.

Acceptance carries ratification conditions, all of them on standards/ver/1.1-draft/RELEASE.md's checklist: conformance fixtures for the dim/dtype requirement, byte-order tests, dtype-aware encoder tests, and a test that exactly one vector carriage is present. A requirement whose whole purpose is that vectors decode correctly is not ratified until something decodes them.

One consequence of acceptance is normative in 1.1 and is stated in the Decision below: the binary dtype's rules are now encoded in the schema, not left in prose.

Context

$defs/embedding requires exactly two members: role and space_id. dim and dtype are optional, and nothing ties either to the referenced space descriptor (Appendix D #15, finding SCH-08). Against the published schema, every one of these passes:

• embedding.dim: 1 against a 768-dimension space;
• embedding.dtype: "int8" against an fp32 space;
• an inline vector of length 3 against dim: 768;
• a vector_b64 whose decoded byte length matches no combination of the declared dim and dtype.

The consequence is not theoretical. A base64 or by-reference vector is an opaque byte string: without a dimension and a dtype it cannot be decoded at all. §6.2 permits three carriages — inline float array, base64 (dtype-tagged, little-endian), and by reference — and only the inline form is self-describing.

Annex E24 resolved the interpretation for 1.0: when dim/dtype are present they MUST equal the descriptor's; when absent the descriptor's values apply; multi-byte dtypes are little-endian; binary packs 8 components per byte with byte length ceil(dim/8). That makes every conformant 1.0 Record decodable, which is the urgent half.

What E24 cannot fix is that the fallback is fragile in exactly the places it matters. The descriptor must be present, correct, and reachable; every consumer must implement the inheritance identically; and any consumer that reads the embedding without resolving its space — the projection case, and the wire case — has nothing to work with. The audit found all three failure modes live: a migration script that decodes every vector as little-endian fp16 regardless of the declared dtype (MIG-02, SCRIPT-DTYPE-ASSUMED); a producer that encodes fp16 while declaring the space's dtype, with nothing checking the two agree (REC-07); and a wire mapper that reinterprets a spec-legal fp32 payload as fp16 (WIRE-11).

Options

(a) Leave both optional; rely on annex E24's inheritance. No compatibility cost, and the fallback is now specified. Every consumer still carries inheritance logic, embeddings remain non-self-describing, and the projection and wire cases remain broken.

(b) Make both REQUIRED for all Records. Every embedding becomes self-describing immediately. This invalidates existing Records that legitimately omitted the fields — a narrowing, and therefore VER 2.0 material under `VERSIONING.md` §2.3, not something 1.1 can ship.

(c) Make both REQUIRED only for Records declaring ver_version: "1.1". New Records are self-describing; old Records stay valid forever. Costs a conditional subschema and leaves the ecosystem with two rules for a while.

(d) Require them only for the non-self-describing carriages (vector_b64, vector_ref), leaving inline vectors alone. Minimal, and targeted precisely at the undecodable case. But it makes the rule depend on which carriage was chosen, so a producer switching carriage acquires a new obligation, and a consumer's expectations vary per embedding within one Record.

Decision

Option (c) — draft-selected, pending ratification.

For Records declaring ver_version: "1.1", embedding.dim and embedding.dtype are REQUIRED.

The gate is the Record's declared ver_version, and how a schema expresses that gate depends on how many versions the schema covers. `VERSIONING.md` §2.2 permits a minor release to promote an optional member to REQUIRED only for records declaring the new ver_version. In a version-pinned schema — one whose ver_version is const — a flat required entry is that gated form: the const already partitions the document set, so every instance the schema can accept declares the new version, and an if/then on the sibling ver_version would test a condition that is true of every one of them. The 1.1 draft schema is version-pinned, so $defs/embedding carries

{ "required": ["role", "space_id", "dim", "dtype"] }

and that is the correct encoding, not a shortcut past one. A multi-version schema — one accepting both "1.0" and "1.1" in one document, which VER does not publish — would need the conditional form:

{ "if":   { "properties": { "ver_version": { "const": "1.1" } } },
  "then": { "properties": { "embeddings": { "items":
            { "required": ["role", "space_id", "dim", "dtype"] } } } } }

Either way the rule is identical: a 1.0 Record is judged by the 1.0.1 schema, which requires neither member, and stays valid in perpetuity.

Supporting rules, all inherited unchanged from annex E24:

1. They must equal the descriptor's values. Stating them is not a licence to contradict the space (VER601, VER602).
2. The stated values are authoritative for decoding, and a consumer that finds them disagreeing with the descriptor MUST reject the embedding rather than choose a winner.
3. Byte-length arithmetic is unchanged: fp32=4, fp16=2, bf16=2, int8=1 bytes per component; binary packs 8 components per byte for a total of ceil(dim/8) bytes. Enforced by VER603–VER605.
4. 1.0 Records are untouched. They may omit both forever; the descriptor's values apply; the profile emits VER606 as a warning, never an error.

binary in 1.1, encoded

A self-describing embedding is only self-describing if dtype: "binary" means something a decoder can act on. In 1.0.x every rule about binary vectors was prose, and the schema accepted a binary space with metric: "cosine" and an inline float array — which is a dtype that describes nothing. For 1.1 the following are schema-encoded, and 1.1 §11.2 carries the normative text:

1. metric MUST be hamming for a binary space, and — the converse — hamming MUST NOT appear on a space whose dtype is not binary.
2. normalization MUST be none. Normalizing a bit vector is not a defined operation.
3. No inline vector array for a binary embedding: the payload is vector_b64 or vector_ref. A ±1 float expansion is a different object from the packed bits, and publishing it under the space's dim misstates the length by a factor of eight.
4. Byte length is ceil(dim/8), packed 8 components per octet, most-significant-bit first within each octet, in ascending component order. This one relates three members, so it stays with VER603–VER605; the schema states it in the vector_b64 description so that the rule and the field are in the same place.
5. A binary space's reference claim is tolerance_hamming_max, in bits, and the cosine tolerances MUST NOT appear on it. The 1.0.x reading — a cosine floor over the ±1 expansion, monotone in Hamming distance — is withdrawn: it is not the quantity a binary retrieval system measures, and it made binary conformance claims incomparable with the systems that consume them.

VER606 is the migration ramp. It fires today against 1.0 Records, costs nothing to heed, and a producer that clears it now needs no change at all when 1.1 ratifies.

Consequences

Good.

• A 1.1 embedding is self-describing. Its bytes can be decoded from the embedding object alone, without resolving a space descriptor.
• The projection cases stop guessing. A wire mapper or an index writer that sees only the embedding has what it needs, which removes the class that produced MIG-02, WIRE-11, and SCRIPT-DTYPE-ASSUMED.
• Producer bugs surface at validation instead of at retrieval. An fp16 encoder under an fp32 declaration is caught by VER602/VER604 rather than by someone noticing that cosine similarities look wrong.
• The compatibility cost is zero for existing Records, by construction.

Bad.

• Duplicated data can disagree. Requiring the values makes contradiction possible where inheritance made it impossible, which is why rule 1 is an error and not a warning. This is a real trade: option (a) has no contradiction because it has no data.
• The gate is invisible in the schema text. A reader of the 1.1 draft schema sees a flat required and has to know that ver_version is const two hundred lines above it to understand that the requirement is version-scoped. The conditional form would have carried its own explanation — at the price of a construct some validators handle inconsistently on a sibling property. The mitigation is documentation, not a construct: this ADR, VERSIONING.md §2.2, and 1.1 §12 each state the equivalence, and the schema's own description says the requirement is for records declaring ver_version: "1.1".
• Two rules coexist for the life of VER 1.x. A consumer supporting both must implement inheritance and the required case. Option (b) avoids this by breaking everyone, which is not a trade worth making.
• Producers that never knew their dtype must now find out. That is arguably the point — a producer that cannot state its vector's dtype does not know what it published — but it is friction, and it will be reported as such.
• The binary encodings reject 1.0.1-valid descriptors. conformance/valid/binary-dtype.json is one: it declares metric: "cosine" on a binary space and a tolerance_cosine, and it fails the 1.1 schema on three pointers. That is the intended effect — those descriptors were unusable in exactly the way the encoding now names — but it is a migration cost and 1.1 §12.1 row 6 states it.

Neutral.

• The wire profile is unaffected by this ADR directly; it pins dtype and normalization with const and enforces neither (WIRE-12). Constraint parity is 1.1 §14.3.
• If ratification reverses to option (a), VER606 stays a warning permanently and the 1.1 schema drops dim and dtype from $defs/embedding's required list. No published artifact changes.