open-brainy/docs/contract-1-ratification.md
David Snelling 742a0b0506 perf(open): answer "are there any entities?" with one directory read
The 7.x-to-8.0 layout probe runs on the open path of every store that does not
yet carry its completion marker — a restore, a store built by an older release
— and asked whether the canonical tree holds anything by LISTING it: a
recursive walk of every file in every entity directory, to learn a boolean. It
now asks the one-level door added for generation discovery, falling back to the
listing on an adapter that lacks it.

Also files a defect found while ratifying the operator set: the VFS builds its
path-prefix filter as `$startsWith`, an operator no engine spelling accepts,
so vfs.searchFiles({ path }) throws INVALID_QUERY on every call that passes a
path. Pre-existing, unrelated to the operator work, and left as a filing —
a path-prefix search needs a design answer, not a spelling correction.
2026-08-28 11:13:24 -07:00

244 lines
12 KiB
Markdown

# Contract 1 — ratification
Open Brainy's answer to the API contract published by the accelerated engine
(`docs/api-contract.md` + `docs/api-contract.json`, contract version 1). Each
item is answered with the code line that proves it, and each promise is stated
as a promise rather than a description.
*Internal engineering document — no frontmatter, not published.*
---
## 1. Contract version — DECLARED
`package.json` carries `"brainyContract": 1`, and the engine states its own:
```ts
export const BRAINY_CONTRACT_VERSION = 1 as const
export function contractVersion(): number { return BRAINY_CONTRACT_VERSION }
```
`src/utils/version.ts`, re-exported from `src/index.ts`. Two engines can now
compare an integer instead of probing prototypes, and a tool can read the
package field without importing the engine. Pinned in
`tests/integration/filter-operator-conformance.test.ts` ("declares its contract
version in code and in package.json") — the code value and the package field
can never drift apart silently.
---
## 2. The REQUIRED / OPTIONAL split — RATIFIED, WITH A COMMITMENT
**Ratified: 41 required doors of 57.** The promise, stated plainly:
> **A REQUIRED door is never removed, never narrowed, and never made optional
> without a MAJOR contract bump.** "Narrowed" includes: refusing an input it
> used to accept, returning less than it used to return, and changing an
> ordering, a cursor encoding, or a refusal's typed code. An OPTIONAL door may
> be added in a minor; an optional door **promoted to required** is a major,
> because a consumer that relied on feature-detecting it now has a hard
> dependency.
Two clarifications this engine attaches, so the promise means the same thing
on both sides:
1. **A refusal is part of the door.** Contract 1 includes not only that
`find({ where })` answers, but that it REFUSES by name for the operators
listed as refused. Turning a refusal into a silent empty answer is a
narrowing, not a relaxation — the same class of change as removing the door.
2. **Deprecation is not removal.** This engine may mark a required door
deprecated in a minor (documented, warned) as long as it keeps working. Only
its removal is a major.
---
## 3. `is` / `isNot` / `greaterEqual` / `lessEqual` — A FINDING AGAINST THE SPEC
**The specification is wrong about these four, and this engine has never
served them.** `docs/filter-operator-conformance.md` (in the accelerated
engine's repository — not editable from here) lists them as served aliases.
The accepted set is defined in one place:
```ts
// src/utils/metadataFilter.ts
const VALUE_OPERATORS = new Set<string>([
'equals', 'eq', 'notEquals', 'ne',
'greaterThan', 'gt', 'greaterThanOrEqual', 'gte',
'lessThan', 'lt', 'lessThanOrEqual', 'lte',
'between', 'oneOf', 'in', 'noneOf',
'contains', 'excludes', 'hasAll', 'length',
'exists', 'missing', 'matches', 'startsWith', 'endsWith'
])
```
25 tokens. None of the four appears; `validateWhereFilter()` raises
`BrainyError('INVALID_QUERY')` naming the bad operator and listing the valid
set, before any index read. A consumer following the spec would have written a
filter this engine rejects outright.
**Action taken here, since the prose lives in the other repository:** the truth
is made machine-checkable rather than re-asserted in another document. The
accepted set is asserted token-for-token in
`tests/integration/filter-operator-conformance.test.ts`, read out of the
engine's own refusal message, and the same set is emitted into
`docs/api-contract.json` (item 8). Diff the manifests; the prose can then be
corrected from a fact.
---
## 4. The serving-withholding invariant list — CONFIRMED IDENTICAL
`index-initialized · durable-state-present · manifest-residency ·
replay-clean · strand-latch`. Confirmed as this engine's list, and confirmed
EXHAUSTIVE for contract 1: these are the only invariants whose failure may
withhold serving. Everything else a health report can fail is a `warn` — it
names damage without closing a door.
The mechanism on this side: `assessProviderHealth()`
(`src/utils/indexReadiness.ts`) treats the provider's own `serving` verdict as
authoritative and verbatim; an UNLEDGERED family never flips a serving provider
to not-ready and never flips a not-serving provider to ready. The read gate
refuses PER FAMILY — a metadata read is never refused by an unserving vector
leg (`src/brainy.ts`, `ensureFamiliesServing`).
**One addition this engine is making, declared here because it changes what a
refusal MEANS:** a provider may now report `rebuildInProgress()` — it is
rebuilding ITSELF, online, and its doors refuse by name with progress until it
is whole. This does not add a withholding invariant (the provider's own
`serving: false` is still what withholds); it adds a REASON attached to that
withholding, so a caller can tell "temporarily closed, opens by itself" from
"broken, needs repairIndex()". Additive, hence a minor.
---
## 5. The compatibility rule — ADOPTED
**Minor = additive. Major = breaking.** Adopted verbatim, with the
announcement duty attached:
> **Every public-surface addition is announced.** The accelerated engine's
> package re-exports this engine's surface by enumeration, so it goes red on
> any new export BY DESIGN — that redness is the announcement mechanism
> working, not a build break to route around.
The mechanics that make this checkable rather than remembered:
`scripts/emit-contract-manifest.mjs --check` fails when the committed
`docs/api-contract.json` no longer matches the built surface. A new export is
therefore a red check with a message naming what to do: re-emit and announce.
---
## 6. The 30 storage seam methods — SUPPORTED SURFACE, COMMITTED
**Committed: every method in `docs/api-contract.md` §15 is supported surface
until Stage 2, and none is removed without a contract major.** They are the
seam the accelerated engine's storage adapter implements and the seam its
reader replaces piece by piece; removing one mid-programme would break a
working pair for no gain.
Two qualifications, both stated so neither side is surprised:
1. **Supported ≠ frozen in behaviour.** A seam method may become FASTER, may
narrate more, and may start refusing an input that was previously an
undefined-behaviour footgun — the last of those is announced as a divergence
here before it ships, not discovered by the other engine.
2. **`counts.json`'s ledger is the one seam value that is not an
enumeration.** See `docs/canonical-layout-ratification.md` §8: only the
all-tier pair carrying `allCountsDerivedBy: 'identity-record'` with
`allCountsSuspect: false` may be subtracted against. That rule is part of
this commitment.
---
## 7. `hasAll` / `noneOf` / `excludes` — SERVED, NOT RATIFIED AS A DIVERGENCE
The accelerated engine was right that it was the correct side, and the
divergence is now closed in the right direction: **this engine serves all three
on the index path.**
The defect underneath was worse than a divergence. The metadata index's
operator switch (`src/utils/metadataIndex.ts`) had **no default case**, so any
operator without a `case` left the field's match set at its initial `[]` and
`find()` returned an empty page. `hasAll`, `noneOf` and `excludes` are
documented, accepted by the validator, and implemented in the in-memory
matcher — and they answered silently wrong through an index-backed find.
- **`hasAll: [a, b]`** — the intersection of each element's posting set. An
empty operand array is vacuously true of every row that HAS the field.
- **`noneOf: [a, b]`** — the complement of the union of their posting sets.
- **`excludes: v`** — the complement of `contains`.
**And the other four are now REFUSED BY NAME rather than answered empty.**
`startsWith`, `endsWith`, `matches` and `length` cannot be evaluated by an
equality/range posting index without reading every row, which is the cost this
path exists to avoid. They raise `BrainyError('INVALID_QUERY')` naming the
operator, the field, and the reason. This matches the accelerated engine's
behaviour for the same four tokens, so the two engines now AGREE on all 25:
| class | tokens |
|---|---|
| served on the index path | between, contains, eq, equals, excludes, exists, greaterThan, greaterThanOrEqual, gt, gte, hasAll, in, lessThan, lessThanOrEqual, lt, lte, missing, ne, noneOf, notEquals, oneOf |
| refused by name | endsWith, length, matches, startsWith |
Pinned in `tests/integration/filter-operator-conformance.test.ts`: the exact
25-token accepted set, the three now served with their real answers (including
an honest zero), and each of the four refusing by name.
**This is a behaviour change for any consumer today calling the four refused
operators through `find({ where })`.** They received an empty page; they now
receive a typed refusal. Converting a wrong answer into a loud refusal is this
engine's own law, and the previous behaviour was not a contract anyone could
have relied on deliberately — but it is a change, and it is named here rather
than discovered.
**`knownDivergences` after this change:** the entry
`served-beyond-baseline` is RESOLVED (both engines serve all three). The entry
`refused-operators-answer-differently` is RESOLVED (both engines refuse the
same four by name). Contract 1 has no remaining operator divergence.
---
## 8. This engine's own manifest — EMITTED
`docs/api-contract.json`, generated by `scripts/emit-contract-manifest.mjs`
from the BUILT surface: the prototype's own methods and accessors, the exported
error classes, the operator sets read out of their single definitions, the
field-addressing vocabulary read out of `src/db/fieldAddressing.ts`, and the
health verdicts. Nothing in it is hand-maintained, so a diff between the two
manifests is a diff between two engines rather than between two authors.
`node scripts/emit-contract-manifest.mjs --check` fails when the committed
manifest is stale — the announcement duty of item 5, made mechanical.
**What the manifest deliberately does NOT carry: requirement marking.** Whether
a door is required is a commitment, not a property of the surface; it is item 2
of this document. The diff the two sides want — "does Open Brainy still expose
every door contract 1 requires?" — is a set-membership check between their
`doors[].name` where `requirement === 'required'` and this manifest's
`doors[].name`.
---
## A defect this work surfaced but did not fix
`src/vfs/VirtualFileSystem.ts` builds a path-prefix filter as
`path: { $startsWith: options.path }` — with a `$` prefix. No operator in this
engine carries a `$`, so `validateWhereFilter()` rejects it with
`INVALID_QUERY` before any index read: **`vfs.searchFiles({ path })` throws
today, on every call that passes a path.** It is pre-existing and unrelated to
the operator work above (the validator refuses it before the index path is
reached), and it is left as a filing rather than fixed here, because the right
answer is a design question — a path-prefix search cannot be served by an
equality/range index, so it needs either a path-segment index or an explicit
in-memory narrow, not a spelling correction.
---
## Summary of what changed in code for this ratification
| item | change |
|---|---|
| 1 | `"brainyContract": 1` in package.json; `contractVersion()` / `BRAINY_CONTRACT_VERSION` exported |
| 3 | the accepted 25-token set asserted from the engine's own refusal message, and emitted into the manifest |
| 7 | `hasAll` / `noneOf` / `excludes` served on the index path; `startsWith` / `endsWith` / `matches` / `length` refused by name instead of answered empty |
| 8 | `scripts/emit-contract-manifest.mjs` + the generated `docs/api-contract.json`, with a `--check` mode |