docs(releases): 10.4.4 consumer notes — correctness and observability, with the performance line stated exactly

A release audit found hostnames, store identities and operational anecdotes in
this branch's commit messages — not trade secrets, but nothing a public
repository's permanent history should carry either. The messages were rewritten
to keep every number and drop every provenance; the rule is written down here
so the next measurement does not have to be caught by an audit.

CONTRIBUTING.md

@@ -64,10 +64,6 @@ see `package.json` for `test:integration`, `test:coverage`, and friends.
   store, measured solo under an exclusive lock" tells a reader everything the
   number depends on; the machine it ran on and whose data it was tell them
   nothing except where somebody's infrastructure lives.
-- **Documents that answer or reference a confidential specification never enter
-  this repository, even summarized.** The public docs describe THIS engine and
-  the published contract, and nothing else — a summary of a private document is
-  still that document's contents.
 
 ## License
 

docs/api-contract.json

@@ -1,6 +1,7 @@
 {
   "contractVersion": 1,
   "engine": "@soulcraftlabs/brainy",
+  "prose": "docs/contract-1-ratification.md",
   "compatibility": {
     "minor": "additive — a new optional door, a new served operator, a new error class; every existing implementation still conforms",
     "major": "breaking — a door removed, an answer narrowed, an ordering law changed, an optional door promoted to required, or an operator moved from served to refused"

docs/canonical-layout-ratification.md

@@ -0,0 +1,415 @@
+# The canonical layout — ratification
+
+Open Brainy's answer to the canonical layout specification written by the
+accelerated engine's reader team. It reads that document's rules against THIS
+repository's writer, at `10.4.3` plus the changes on
+`next/open-lazy-open-and-counts`, and for each one says **confirmed**,
+**corrected**, or **changed under you** — with the line of code that proves it.
+
+The specification was written against `10.4.1`. Three of its rules have moved
+since, and one of its open questions has already been answered by a fix that
+shipped in `10.4.2`. Those are called out first, because a reader
+implementation built to the draft would be wrong about them today.
+
+*Internal engineering document — no frontmatter, not published.*
+
+---
+
+## What moved since the draft was written
+
+| # | Draft says | Today | Where |
+|---|---|---|---|
+| A | `scanCanonicalEntities()` counts id DIRECTORIES, so a legacy ledger's derivation includes scars and orphans (§4, open question 6) | **CORRECTED IN 10.4.2.** The scan counts one entity per IDENTITY RECORD, sharing its predicate with the orphan pruner | `src/storage/adapters/fileSystemStorage.ts` `scanCanonicalEntities()` / `hasMetadataContentLeg()` |
+| B | An already-derived legacy denominator is persisted "once and never rescanned" (§4, §8) | **CORRECTED on this branch.** A ledger without the `identity-record` stamp is marked suspect at open AND re-derived honestly in the background, then persisted stamped | `scheduleCountLedgerDerivation()`, same file |
+| C | `counts.json` has a zero-byte window because `persistCounts()` uses a bare `fs.writeFile` (§8.1, open question 5) | **FIXED on this branch.** Written temp+rename like every other object | `persistCounts()`, same file |
+
+Everything else below stands as the draft states it, except where marked.
+
+---
+
+## §1 — the tree is a projection, not the source of truth
+
+**CONFIRMED.** The generation log under `_generations/` is canonical and the
+entity tree is a materialization of it; `_system/generation.json[.gz]` carries
+`{ generation, updatedAt }` for the projection. `src/db/generationStore.ts`
+owns the counter and its manifest; `readBrainFormat` / the entity-tree stamp
+are read at open in `src/brainy.ts`'s generation-store phase.
+
+Two walks minutes apart may legitimately disagree, and a caller that needs a
+point-in-time answer must pin the generation. Confirmed as a rule of this
+engine, not merely of the reader.
+
+---
+
+## §2 — paths and sharding
+
+**CONFIRMED, all three clauses.**
+
+- Two lowercase hex characters, 256 buckets, and an id that normalises to 32
+  hex characters buckets by its first byte: `src/storage/sharding.ts`
+  `getShardId()` — `const normalized = id.toLowerCase().replace(/-/g, '')`,
+  `if (/^[0-9a-f]{32}$/.test(normalized)) return normalized.substring(0, 2)`.
+- Every other id hashes: `hashToShardId(id)`, FNV-1a over **UTF-16 code
+  units** (`id.charCodeAt(i)`), folded to the low byte. The UTF-16 detail is
+  load-bearing above the BMP and is part of the on-disk contract — changing it
+  relocates every application-id record.
+- `hnsw/` and `metadata/` under `entities/<kind>/` are vestigial and the write
+  path never populates them. Every scan in this repository excludes them with
+  the same two-hex test (`/^[0-9a-f]{2}$/i`) — see `scanCanonicalEntities()`
+  and `scanVectoredNounCount()`. **A reader that counts directory entries
+  reports 258 shards where there are 256** is correct and worth keeping in the
+  spec.
+
+---
+
+## §3 — encodings
+
+**CONFIRMED.**
+
+- The compressed spelling APPENDS: `const compressedPath = \`${fullPath}.gz\``
+  (`fileSystemStorage.ts`, the object read and write paths). Deriving it by
+  replacing the extension finds a file that never exists.
+- Read order is `.gz` first, then plain — same call sites.
+- The three outcomes are distinct in this engine too: absence is `null`, an
+  undecodable body raises `TornRecordError` (`src/storage/tornRecordError.ts`,
+  `registerTornRecordEncounter`), and an IO fault is rethrown unreshaped.
+- The legacy exception the draft names — a torn `.gz` with a decodable plain
+  twin returns the recovered plain object, loudly — is real and deliberate; the
+  code comment at the read path states it in those terms.
+
+**Divergence (torn rows): RATIFIED AS PERMANENT for the current contract.**
+This engine's paginated walks skip-and-count a torn row so enumeration heals
+past it; the reader raises with a resume cursor. Both are defensible and they
+are not reconcilable by picking one: an enumeration that raises cannot serve a
+heal, and a heal that skips silently is how corruption becomes invisible. The
+reader's `resume_cursor` gives a caller the skip behaviour deliberately, which
+is the better shape. **This engine will not change its walks to raise** — its
+walks exist to heal — and the two enumerations are therefore allowed to report
+different populations over a DAMAGED store, never over a healthy one. Any
+consumer comparing the two must first establish the store is not damaged.
+
+---
+
+## §4 — the population law
+
+**CONFIRMED as the law. CORRECTED as a description of this engine's code.**
+
+The law is right and is this engine's own: *the identity record is the
+population; the id directory is not.* `pruneOrphanedEntities()` states it and
+`hasMetadataContentLeg()` is the single predicate both it and the count scan
+use, so the two agree by construction:
+
+```ts
+private hasMetadataContentLeg(legs: string[]): boolean {
+  return legs.some((f) => f.startsWith('metadata.json'))
+}
+```
+
+**The draft's finding that `scanCanonicalEntities()` counts directories is out
+of date.** It was true of `10.4.1`, which the draft was written against; it was
+fixed in `10.4.2` ("derive the canonical count ledger from identity records,
+stamp the derivation rule, and mark legacy-derived ledgers suspect at load").
+The scan now `continue`s on any container with no metadata content leg.
+
+**But the draft's MEASUREMENTS remain valid, and the defect they name was only
+half-fixed until this branch.** A ledger DERIVED under the old rule was
+adopted at load with its wrong numbers and merely flagged `allCountsSuspect`,
+and nothing corrected it short of an operator running `repairIndex()`. That is
+why the same frozen archive still reads 14,231 or 14,081 against 14,056
+identity records depending on which copy you open, and why a graph heal
+subtracted 72,679 walked rows from a 72,729 denominator and reported
+`remaining: 50` — the store's 50 verb scar directories. The heal was behaving
+correctly against a lying denominator.
+
+**On this branch the ledger heals itself**: a load that finds no
+`identity-record` stamp marks the scalars suspect, schedules an honest
+derivation in the BACKGROUND (never blocking the open — the same walk is part
+of why a 24,898-id store opened in silence), and persists the corrected
+scalars stamped. A derivation that raced a write refuses to stamp its number
+and leaves the ledger suspect, naming `repairIndex()` as the door that
+recounts under a barrier. Pinned in
+`tests/integration/count-ledger-identity-record.test.ts`, including two copies
+of one archive deriving the same number.
+
+**Answer to open question 6, in full:**
+
+1. *Should the derivation require an identity leg?* **Yes, and it does** — since
+   `10.4.2`, sharing the pruner's predicate.
+2. *Should already-persisted denominators on affected stores be re-derived?*
+   **Yes, and they now are, automatically, in the background.** An operator no
+   longer has to know.
+3. *Which scalar may a consumer subtract against?* See the next section — the
+   answer is narrower than "the corrected one".
+
+---
+
+## §8 / open question 6 — WHICH SCALAR A CONSUMER MAY SUBTRACT AGAINST
+
+The specification asked this to be stated in the ratification. Stated:
+
+| scalar | what it is | safe to subtract an enumeration against? |
+|---|---|---|
+| `totalNounCount` / `totalVerbCount` | the **counted tiers only** (`isCountedVisibility` excludes `internal` and `system`) | **NO.** It is routinely far below the live population by design. A full-tier walk measured against it reads permanently over. |
+| `totalNounCountAll` / `totalVerbCountAll` **with `allCountsDerivedBy: 'identity-record'` and `allCountsSuspect: false`** | the all-tier identity-record population | **YES.** This is the only pair a coverage verdict may use. |
+| the same scalars **without the stamp, or with `allCountsSuspect: true`** | an unverified number: either derived under the old container rule, or made unprovable by a delete that could not read its record | **NO.** Read the flag, refuse the verdict, and say so. On this branch the engine is already correcting it behind you; wait for the correction rather than subtracting against the interim. |
+| `totalVectoredNounCount` | nouns holding a real, non-empty, non-zero-norm vector, derived by reading record CONTENT | **YES for the vector family only.** It never had the container defect. It shares `allCountsSuspect`, so the same flag rule applies. |
+
+`getCanonicalCounts()` (`src/storage/adapters/baseStorageAdapter.ts`) returns
+all of them with the suspect flag in one O(1) call, and the flag is the whole
+contract: **a consumer that ignores `suspect` is computing a verdict the engine
+has told it not to compute.**
+
+The draft's §8 rules are otherwise **CONFIRMED**: the ledger is counted-tier,
+an absent key is a legacy file and not a zero, and an absent `counts.json` is a
+real condition (this engine rebuilds it from disk on the next open — see
+`initializeCountsFromDisk()`, which on this branch narrates that it is doing so
+and why it cannot be backgrounded).
+
+**Open question 3 — the ledger drifted from the public tier by 3 nouns and 1
+verb.** Not ratified as expected drift. The counted-tier scalars are maintained
+incrementally and can be left inflated by a delete whose decrement was skipped;
+that is exactly what `allCountsSuspect` exists to record and what
+`rebuildTypeCounts()` (the sanctioned recount, `src/storage/baseStorage.ts`)
+corrects. A drift of 3 against 8,663 on a store with a long delete history is
+consistent with that mechanism, not with a fresh defect — but it is a REAL
+inaccuracy, not a definition, and the cure is a recount. **Finding for the
+spec: do not describe counted-tier drift as normal; describe it as an
+uncorrected incremental counter.**
+
+---
+
+## §5.1 — the identity leg
+
+**CONFIRMED, every clause.**
+
+- Two shapes coexist in one store: `_fmt: 2` nested-bag, and the legacy flat
+  shape. `src/types/reservedFields.ts` — `METADATA_RECORD_FORMAT_KEY = '_fmt'`,
+  `NESTED_BAG_FORMAT`.
+- **The stamp alone does not decide it.** `isNestedBagRecord()` requires the
+  stamp AND `typeof record.metadata === 'object'` AND non-null AND
+  `!Array.isArray(record.metadata)`. A reader that trusts the stamp alone
+  mis-splits a pre-law record carrying a user field named `metadata`. The draft
+  is right to make this a rule.
+- Engine fields sit at top level in BOTH shapes, so extraction is
+  shape-independent: `RESERVED_ENTITY_FIELDS` / `RESERVED_RELATIONSHIP_FIELDS`.
+  The entity type is under `noun` (verbs: `verb`), never `type`.
+- **No BigInt at this boundary.** Confirmed — every numeric in a metadata
+  record is a JSON number.
+
+**Divergence (timestamps): RATIFIED IN THE READER'S FAVOUR, and this engine
+should follow.** The draft is correct that this engine substitutes `Date.now()`
+where a stored timestamp is unusable. That is inventing data: a row whose
+`updatedAt` could not be read is not a row that was updated now. The reader's
+`null` is right. **This is a finding against this engine, filed here rather
+than fixed on this branch** — changing hydration's timestamp fallback touches
+every read path and belongs in its own change with its own pins, not in a
+branch about opens and counters. Until then, a consumer must treat a timestamp
+equal to read-time as suspect.
+
+---
+
+## §5.2 — the vector leg
+
+**CONFIRMED, every clause**, including the three that are easiest to get wrong:
+
+- It is **JSON**, not an mmap-able buffer; raw blobs live in `_blobs/` behind
+  `getBinaryBlobPath()` — a different door entirely.
+- **The five-way vector state** is this engine's own law: `absent-leg`,
+  `absent-field`, `empty` (deferred embed — `vector: []` written at `add()`
+  time), `zero-norm`, `real`. `isZeroNormVector()` (`src/utils/distance.ts`) is
+  the shared predicate, and `scanVectoredNounCount()` applies exactly it. A
+  zero-norm vector **is not a vector** — the 10.4.2 unvector work is precisely
+  this rule, canonical side included.
+- **`connections` on disk is not index state.** Confirmed. Nothing may infer
+  that a store is indexed, or how, from that field or from the vestigial
+  `hnsw/` directories.
+
+**Open question 4 — vector legs carrying stale identity copies.** **CONFIRMED
+NEVER AUTHORITATIVE.** Type, subtype and visibility are read from the identity
+leg only; the copies in a vector leg are residue from an older writer. This
+engine will not start reading them and any reader that does will disagree with
+it on rows the two writers touched at different times. Whether Stage 2 drops
+them is the accelerated engine's format decision, not this engine's — from
+here, dropping them is safe.
+
+**The `float_roundtrip` finding is accepted and worth restating as a rule for
+any future reader in any language:** the leg is JSON text, so every component
+is a double, and a decimal has exactly one nearest `f64`. A parser one ULP off
+builds a different index — different distances, different neighbours — and
+makes any parity claim between two engines false. This engine gets it right
+only because `JSON.parse` is correctly rounded by specification; nothing in
+this repository would have caught the divergence, and the conformance suite's
+whole-vector deep-equal is the right instrument.
+
+---
+
+## §6 — verb endpoints
+
+**CONFIRMED, exactly as stated, and it is a defect of shape rather than of
+data.**
+
+`saveVerb_internal()` (`src/storage/baseStorage.ts`) writes the whole
+`HNSWVerb` — `{ id, vector, connections, verb, sourceId, targetId }` — to
+`entities/verbs/<shard>/<id>/vectors.json`, and `saveVerbMetadata()` writes the
+engine scalars and user bag to `metadata.json` with **no endpoints**. `getVerb()`
+reassembles the two.
+
+The consequences the draft draws are all correct, including the one that costs
+the most: **a graph heal cannot be driven from identity records alone**, so
+every edge's vector leg is read purely for its structure even though a verb's
+`vector` is always `[]`. This engine confirms the measurement in principle —
+every verb it has ever written carries its endpoints in the vector leg and none
+in the identity leg.
+
+The door contract that returns `(id, sourceId, targetId, verb, subtype)`
+regardless of which leg holds the bytes is the right shape and this engine
+endorses it as the migration seam. **Commitment: when Stage 2 moves endpoints
+onto the verb record, this engine's reader side must accept BOTH placements for
+the life of contract 1** — a store written by either engine must be readable by
+the other.
+
+---
+
+## §7 — `_system/`
+
+**CONFIRMED**, with one clause now needing a footnote.
+
+- `_system/` mixes persisted state with live protocol and must never be
+  enumerated as data. `idx/` and `family-stamps/` are subtrees.
+- `tx-log.jsonl` is mutated in place and must be byte-copied, not hard-linked,
+  by a snapshot.
+- `<root>/locks/` is live coordination — `_writer.lock` and the flush-request
+  protocol. Reading them as records is meaningless; writing or deleting them
+  interferes with a live writer. **Footnote: this branch adds one more file
+  there, `_writer.close`** — the clean-close record naming the lock generation
+  a writer released, consumed by the next claim. Same rule applies: it is
+  protocol, not data. A reader must ignore it, and must not treat its presence
+  or absence as a fact about the store's contents.
+- **The encoding split is real and remains real:** everything in `_system/`
+  goes through the compressing writer and is `.gz` **except `counts.json`**,
+  which is plain JSON. This branch made that write ATOMIC but deliberately did
+  NOT make it `.gz` — the ledger is the one file an operator reads with `cat`
+  during an incident, and a reader implementation already handles the
+  exception. Try both spellings, as the draft says.
+
+---
+
+## §8.1 — the zero-byte ledger window
+
+**ACCEPTED AS A DEFECT AND FIXED ON THIS BRANCH.**
+
+Cause 1 is confirmed exactly as written: `persistCounts()` used a bare
+`fs.writeFile`, which truncates first, so the ledger was empty for the whole
+write while every other object in the store was written atomically. It now goes
+through the same temp+rename path (`writeFileAtomic`), so a reader sees the old
+ledger or the new one and never neither.
+
+Cause 2 — "the final write is not awaited by the closing path" — **NOT
+REPRODUCED at 10.4.3**, and the mechanism named does not exist here: the
+filesystem adapter's count persist is write-through, not debounced
+(`scheduleCountPersist()` sets the flag and immediately `await`s
+`flushCounts()`), and `close()` awaits `flushCounts()` in its first flush
+phase. The ~750 ms the draft measured is more likely the tail of an in-flight
+write-through persist observed through the truncation window of cause 1, which
+is now closed. **Finding: re-measure on 10.4.4 before keeping cause 2 in the
+spec.**
+
+**Answer to open question 5:** yes, fixed; and yes, a reader should keep
+classifying a zero-byte ledger as `TORN_RECORD` — the classification is right
+on its own terms (a file that exists and holds nothing IS torn), it is the only
+honest reading of a store damaged by an older release, and stores written by
+every version before 10.4.4 can still carry one.
+
+Pinned here by `tests/integration/count-ledger-identity-record.test.ts` —
+40 consecutive persists watched at 1 ms and never once unparseable.
+
+---
+
+## §9 — the ordering law
+
+**CONFIRMED, including the two clauses most likely to be "simplified" by a
+future implementer.**
+
+- Shard `00` → `ff` ascending, then id ascending in **UTF-16 code-unit order** —
+  this engine's `.sort()` with an `a.id < b.id` comparator, which is exactly
+  UTF-16 code-unit order, not UTF-8 byte order and not code-point order. They
+  diverge above the BMP.
+- Cursor tokens: `cn1:<shard>:<id>` and `cv1:<shard>:<id>`, shard in DECIMAL
+  (it is the loop index, not the directory name), id last so an id containing
+  `:` survives the round trip. `src/storage/baseStorage.ts` —
+  `` return `cn1:${shard}:${id}` `` and `` return `cv1:${shard}:${id}` ``.
+- A supplied-but-undecodable cursor must FAIL, never restart at offset 0.
+  Confirmed as a rule: silently restarting turns a `while (hasMore)` loop into
+  an unbounded one.
+- The stability properties (a consistent prefix per shard, not a snapshot;
+  appends behind the cursor missed; deletes ahead take effect) are this
+  engine's own behaviour, confirmed.
+
+---
+
+## §10 — read depth
+
+**CONFIRMED as a design, with one clause this engine wants on the record:**
+*read depth bounds what corruption can be seen.* An ids-only enumeration opens
+nothing and therefore reports a clean population over a store of undecodable
+bodies. That is the honest consequence of reading exactly what was asked for.
+A caller asking "is this store healthy?" from the cheap level is asking the
+wrong door — health needs a decoding read. This engine's health reporting
+follows the same rule and should say so as plainly.
+
+---
+
+## §11 — cost
+
+**NOT RATIFIED — NOT THIS ENGINE'S TO RATIFY.** The comparative walls are
+measurements of two implementations on one box; nothing in this repository can
+confirm or refute them. Two observations that ARE this engine's:
+
+1. The structural explanation offered for the verb gap is consistent with this
+   engine's code: the noun walk hydrates in batches, the verb walk does not,
+   and a cursored resume re-lists and re-sorts the cursor's shard on every
+   page. That is a real asymmetry in `src/storage/baseStorage.ts` and it is
+   fair to name it.
+2. The draft's own honesty about the applier holding 98% of the verb wall after
+   fan-out is the part worth keeping. **Nobody may read "8 workers" as "8×".**
+
+---
+
+## §12 — Stage 1
+
+**NOTED, NOT RATIFIED.** Which read path the accelerated engine's walks use is
+its own decision. Two things this engine confirms because they are claims about
+THIS code:
+
+- **The membership-predicate correction is right.** This engine's membership
+  is the IDENTITY RECORD: an id directory holding a vector leg and no identity
+  record is enumerated by neither `getNouns()` nor
+  `getNounIdsWithPagination()` and is absent from `getCanonicalCounts()`; an
+  identity-only row IS enumerated and IS counted. The draft's earlier statement
+  that the doors key on `vectors.json` was wrong and its correction is accepted.
+- **Read-your-writes is confirmed.** There is no write-behind buffer between an
+  awaited `saveNoun()` / `saveNounMetadata()` and the bytes on disk; the leg is
+  `stat`-able before the returned promise resolves. **Commitment: this engine
+  will not introduce a write-behind buffer in the filesystem adapter without
+  declaring a flush contract alongside it** — the law "anything not yet awaited
+  is not yet canonical" is only useful if the converse holds.
+
+---
+
+## Findings this ratification files back
+
+1. **§4 / open question 6 is out of date** — the directory-counting scan was
+   fixed in 10.4.2. Re-read the spec's §4 against 10.4.3 before Stage 2 builds
+   on it. The MEASUREMENTS stay valid; the code claim does not.
+2. **§8.1 cause 2 is not reproducible at 10.4.3** — the count persist is
+   write-through and `close()` awaits it. Re-measure before republishing.
+3. **§8's counted-tier drift (open question 3) should not be described as
+   expected** — it is an uncorrected incremental counter, and `repairIndex()`
+   is its cure.
+4. **The timestamp divergence is a defect on THIS engine's side**, not a
+   difference of opinion: substituting `Date.now()` for an unreadable
+   timestamp invents data. Filed, not fixed on this branch.
+5. **`locks/_writer.close` is new** — a reader must ignore it like every other
+   file under `locks/`.
+6. **`counts.json` is now written atomically and stays plain JSON** — the
+   `_system/` encoding exception is deliberate and permanent for contract 1.

docs/contract-1-ratification.md

@@ -0,0 +1,244 @@
+# Contract 1 — ratification
+
+Open Brainy's answer to the API contract published by the accelerated engine
+(`docs/api-contract.md` + `docs/api-contract.json`, contract version 1). Each
+item is answered with the code line that proves it, and each promise is stated
+as a promise rather than a description.
+
+*Internal engineering document — no frontmatter, not published.*
+
+---
+
+## 1. Contract version — DECLARED
+
+`package.json` carries `"brainyContract": 1`, and the engine states its own:
+
+```ts
+export const BRAINY_CONTRACT_VERSION = 1 as const
+export function contractVersion(): number { return BRAINY_CONTRACT_VERSION }
+```
+
+`src/utils/version.ts`, re-exported from `src/index.ts`. Two engines can now
+compare an integer instead of probing prototypes, and a tool can read the
+package field without importing the engine. Pinned in
+`tests/integration/filter-operator-conformance.test.ts` ("declares its contract
+version in code and in package.json") — the code value and the package field
+can never drift apart silently.
+
+---
+
+## 2. The REQUIRED / OPTIONAL split — RATIFIED, WITH A COMMITMENT
+
+**Ratified: 41 required doors of 57.** The promise, stated plainly:
+
+> **A REQUIRED door is never removed, never narrowed, and never made optional
+> without a MAJOR contract bump.** "Narrowed" includes: refusing an input it
+> used to accept, returning less than it used to return, and changing an
+> ordering, a cursor encoding, or a refusal's typed code. An OPTIONAL door may
+> be added in a minor; an optional door **promoted to required** is a major,
+> because a consumer that relied on feature-detecting it now has a hard
+> dependency.
+
+Two clarifications this engine attaches, so the promise means the same thing
+on both sides:
+
+1. **A refusal is part of the door.** Contract 1 includes not only that
+   `find({ where })` answers, but that it REFUSES by name for the operators
+   listed as refused. Turning a refusal into a silent empty answer is a
+   narrowing, not a relaxation — the same class of change as removing the door.
+2. **Deprecation is not removal.** This engine may mark a required door
+   deprecated in a minor (documented, warned) as long as it keeps working. Only
+   its removal is a major.
+
+---
+
+## 3. `is` / `isNot` / `greaterEqual` / `lessEqual` — A FINDING AGAINST THE SPEC
+
+**The specification is wrong about these four, and this engine has never
+served them.** `docs/filter-operator-conformance.md` (in the accelerated
+engine's repository — not editable from here) lists them as served aliases.
+The accepted set is defined in one place:
+
+```ts
+// src/utils/metadataFilter.ts
+const VALUE_OPERATORS = new Set<string>([
+  'equals', 'eq', 'notEquals', 'ne',
+  'greaterThan', 'gt', 'greaterThanOrEqual', 'gte',
+  'lessThan', 'lt', 'lessThanOrEqual', 'lte',
+  'between', 'oneOf', 'in', 'noneOf',
+  'contains', 'excludes', 'hasAll', 'length',
+  'exists', 'missing', 'matches', 'startsWith', 'endsWith'
+])
+```
+
+25 tokens. None of the four appears; `validateWhereFilter()` raises
+`BrainyError('INVALID_QUERY')` naming the bad operator and listing the valid
+set, before any index read. A consumer following the spec would have written a
+filter this engine rejects outright.
+
+**Action taken here, since the prose lives in the other repository:** the truth
+is made machine-checkable rather than re-asserted in another document. The
+accepted set is asserted token-for-token in
+`tests/integration/filter-operator-conformance.test.ts`, read out of the
+engine's own refusal message, and the same set is emitted into
+`docs/api-contract.json` (item 8). Diff the manifests; the prose can then be
+corrected from a fact.
+
+---
+
+## 4. The serving-withholding invariant list — CONFIRMED IDENTICAL
+
+`index-initialized · durable-state-present · manifest-residency ·
+replay-clean · strand-latch`. Confirmed as this engine's list, and confirmed
+EXHAUSTIVE for contract 1: these are the only invariants whose failure may
+withhold serving. Everything else a health report can fail is a `warn` — it
+names damage without closing a door.
+
+The mechanism on this side: `assessProviderHealth()`
+(`src/utils/indexReadiness.ts`) treats the provider's own `serving` verdict as
+authoritative and verbatim; an UNLEDGERED family never flips a serving provider
+to not-ready and never flips a not-serving provider to ready. The read gate
+refuses PER FAMILY — a metadata read is never refused by an unserving vector
+leg (`src/brainy.ts`, `ensureFamiliesServing`).
+
+**One addition this engine is making, declared here because it changes what a
+refusal MEANS:** a provider may now report `rebuildInProgress()` — it is
+rebuilding ITSELF, online, and its doors refuse by name with progress until it
+is whole. This does not add a withholding invariant (the provider's own
+`serving: false` is still what withholds); it adds a REASON attached to that
+withholding, so a caller can tell "temporarily closed, opens by itself" from
+"broken, needs repairIndex()". Additive, hence a minor.
+
+---
+
+## 5. The compatibility rule — ADOPTED
+
+**Minor = additive. Major = breaking.** Adopted verbatim, with the
+announcement duty attached:
+
+> **Every public-surface addition is announced.** The accelerated engine's
+> package re-exports this engine's surface by enumeration, so it goes red on
+> any new export BY DESIGN — that redness is the announcement mechanism
+> working, not a build break to route around.
+
+The mechanics that make this checkable rather than remembered:
+`scripts/emit-contract-manifest.mjs --check` fails when the committed
+`docs/api-contract.json` no longer matches the built surface. A new export is
+therefore a red check with a message naming what to do: re-emit and announce.
+
+---
+
+## 6. The 30 storage seam methods — SUPPORTED SURFACE, COMMITTED
+
+**Committed: every method in `docs/api-contract.md` §15 is supported surface
+until Stage 2, and none is removed without a contract major.** They are the
+seam the accelerated engine's storage adapter implements and the seam its
+reader replaces piece by piece; removing one mid-programme would break a
+working pair for no gain.
+
+Two qualifications, both stated so neither side is surprised:
+
+1. **Supported ≠ frozen in behaviour.** A seam method may become FASTER, may
+   narrate more, and may start refusing an input that was previously an
+   undefined-behaviour footgun — the last of those is announced as a divergence
+   here before it ships, not discovered by the other engine.
+2. **`counts.json`'s ledger is the one seam value that is not an
+   enumeration.** See `docs/canonical-layout-ratification.md` §8: only the
+   all-tier pair carrying `allCountsDerivedBy: 'identity-record'` with
+   `allCountsSuspect: false` may be subtracted against. That rule is part of
+   this commitment.
+
+---
+
+## 7. `hasAll` / `noneOf` / `excludes` — SERVED, NOT RATIFIED AS A DIVERGENCE
+
+The accelerated engine was right that it was the correct side, and the
+divergence is now closed in the right direction: **this engine serves all three
+on the index path.**
+
+The defect underneath was worse than a divergence. The metadata index's
+operator switch (`src/utils/metadataIndex.ts`) had **no default case**, so any
+operator without a `case` left the field's match set at its initial `[]` and
+`find()` returned an empty page. `hasAll`, `noneOf` and `excludes` are
+documented, accepted by the validator, and implemented in the in-memory
+matcher — and they answered silently wrong through an index-backed find.
+
+- **`hasAll: [a, b]`** — the intersection of each element's posting set. An
+  empty operand array is vacuously true of every row that HAS the field.
+- **`noneOf: [a, b]`** — the complement of the union of their posting sets.
+- **`excludes: v`** — the complement of `contains`.
+
+**And the other four are now REFUSED BY NAME rather than answered empty.**
+`startsWith`, `endsWith`, `matches` and `length` cannot be evaluated by an
+equality/range posting index without reading every row, which is the cost this
+path exists to avoid. They raise `BrainyError('INVALID_QUERY')` naming the
+operator, the field, and the reason. This matches the accelerated engine's
+behaviour for the same four tokens, so the two engines now AGREE on all 25:
+
+| class | tokens |
+|---|---|
+| served on the index path | between, contains, eq, equals, excludes, exists, greaterThan, greaterThanOrEqual, gt, gte, hasAll, in, lessThan, lessThanOrEqual, lt, lte, missing, ne, noneOf, notEquals, oneOf |
+| refused by name | endsWith, length, matches, startsWith |
+
+Pinned in `tests/integration/filter-operator-conformance.test.ts`: the exact
+25-token accepted set, the three now served with their real answers (including
+an honest zero), and each of the four refusing by name.
+
+**This is a behaviour change for any consumer today calling the four refused
+operators through `find({ where })`.** They received an empty page; they now
+receive a typed refusal. Converting a wrong answer into a loud refusal is this
+engine's own law, and the previous behaviour was not a contract anyone could
+have relied on deliberately — but it is a change, and it is named here rather
+than discovered.
+
+**`knownDivergences` after this change:** the entry
+`served-beyond-baseline` is RESOLVED (both engines serve all three). The entry
+`refused-operators-answer-differently` is RESOLVED (both engines refuse the
+same four by name). Contract 1 has no remaining operator divergence.
+
+---
+
+## 8. This engine's own manifest — EMITTED
+
+`docs/api-contract.json`, generated by `scripts/emit-contract-manifest.mjs`
+from the BUILT surface: the prototype's own methods and accessors, the exported
+error classes, the operator sets read out of their single definitions, the
+field-addressing vocabulary read out of `src/db/fieldAddressing.ts`, and the
+health verdicts. Nothing in it is hand-maintained, so a diff between the two
+manifests is a diff between two engines rather than between two authors.
+
+`node scripts/emit-contract-manifest.mjs --check` fails when the committed
+manifest is stale — the announcement duty of item 5, made mechanical.
+
+**What the manifest deliberately does NOT carry: requirement marking.** Whether
+a door is required is a commitment, not a property of the surface; it is item 2
+of this document. The diff the two sides want — "does Open Brainy still expose
+every door contract 1 requires?" — is a set-membership check between their
+`doors[].name` where `requirement === 'required'` and this manifest's
+`doors[].name`.
+
+---
+
+## A defect this work surfaced but did not fix
+
+`src/vfs/VirtualFileSystem.ts` builds a path-prefix filter as
+`path: { $startsWith: options.path }` — with a `$` prefix. No operator in this
+engine carries a `$`, so `validateWhereFilter()` rejects it with
+`INVALID_QUERY` before any index read: **`vfs.searchFiles({ path })` throws
+today, on every call that passes a path.** It is pre-existing and unrelated to
+the operator work above (the validator refuses it before the index path is
+reached), and it is left as a filing rather than fixed here, because the right
+answer is a design question — a path-prefix search cannot be served by an
+equality/range index, so it needs either a path-segment index or an explicit
+in-memory narrow, not a spelling correction.
+
+---
+
+## Summary of what changed in code for this ratification
+
+| item | change |
+|---|---|
+| 1 | `"brainyContract": 1` in package.json; `contractVersion()` / `BRAINY_CONTRACT_VERSION` exported |
+| 3 | the accepted 25-token set asserted from the engine's own refusal message, and emitted into the manifest |
+| 7 | `hasAll` / `noneOf` / `excludes` served on the index path; `startsWith` / `endsWith` / `matches` / `length` refused by name instead of answered empty |
+| 8 | `scripts/emit-contract-manifest.mjs` + the generated `docs/api-contract.json`, with a `--check` mode |

scripts/emit-contract-manifest.mjs

@@ -10,8 +10,8 @@
  * between two engines, never between two authors.
  *
  * Requirement marking (required / optional per door) is NOT derivable from the
- * surface — it is a commitment, recorded with the contract's owner rather than
- * here. This manifest carries the surface; the promise lives with the contract.
+ * surface; it is a commitment, and it lives in docs/contract-1-ratification.md.
+ * This manifest carries the surface; that document carries the promise.
  *
  * Usage: node scripts/emit-contract-manifest.mjs [--check]
  *   --check exits non-zero when the committed manifest is stale.
@@ -68,6 +68,7 @@ const servedOnIndex = accepted.filter((op) => !refusedByIndex.includes(op))
 const manifest = {
   contractVersion: versionModule.contractVersion(),
   engine: '@soulcraftlabs/brainy',
+  prose: 'docs/contract-1-ratification.md',
   compatibility: {
     minor:
       'additive — a new optional door, a new served operator, a new error class; every existing implementation still conforms',