docs(release): the 10.4.0 entry, the index-health concept doc, and the API surfaces — written from the tree, not the plan
Some checks failed
CI / Node 22 (push) Successful in 12m19s
CI / Node 24 (push) Successful in 12m16s
CI / Integration + conformance (Node 22) (push) Failing after 15m12s
CI / Bun (latest) (push) Successful in 12m24s

This commit is contained in:
David Snelling 2026-08-25 10:02:25 -07:00
parent b9ba50fbec
commit 8cced871a0
8 changed files with 454 additions and 63 deletions

View file

@ -31,6 +31,90 @@ is sometimes cited as a 7.x removal — those methods never existed on 7.x; the
--- ---
## v10.4.0 — 2026-08-25 (the health report has a name)
Three related cures, one root cause: an index deciding whether it could be trusted
by sampling itself instead of by exact accounting. This release replaces every
sampled self-probe with ledger-derived truth, and a read against an unhealthy index
now refuses loudly instead of guessing.
- **The canonical count ledger.** Storage now tracks two scalars per family
(nouns/verbs) on the write path: the user-facing `counted` total — unchanged,
still what `getNounCount()` / `getVerbCount()` return — and a new ALL-visibility
`all` total covering every tier, the real denominator a derived index's own
coverage math needs. The unfiltered storage-level `totalCount` returned by
`getNouns()` / `getVerbs()` is now this unclamped ALL scalar; previously it could
only ever move up (`Math.max(scalar, scanned)`), so an inflated counter could
never self-correct. A delete that cannot prove the record it removed actually
existed (no canonical read, no prior image available) no longer decrements on
faith — it marks the ledger `suspect` (narrated once per session) instead of
silently drifting, and the next `repairIndex()` clears the flag with a real
recount.
- **One contract for a throwing health probe.** A provider's `validateInvariants()`
is documented to never throw — but if one does anyway (a bug, a transient fault),
it is now read the same way everywhere: `heal: 'none'`, the error named in the
report, never synthesized into a rebuild trigger and never swallowed into "looks
fine." A flaky check can no longer buy itself a rebuild. `repairIndex()`'s
per-family receipt also gains `missing` (an exact count plus a capped id sample),
`rebuilt` (a full rebuild ran, vs. an incremental heal), and `reason`.
- **The named health report; reads refuse instead of rebuilding.** Any index
provider may now expose a synchronous, O(1) `healthReport()` — composed from the
provider's own exact ledgers, never a sample — and this is the one signal
Brainy's read gate trusts. The first-query lazy-build path is gone: `brain.init()`
now runs every needed rebuild to completion before it returns, always, regardless
of dataset size. A read that lands on a provider whose health report says it
isn't serving throws a typed error instead of triggering a rebuild mid-query —
`GraphIndexNotReadyError`, `MetadataIndexNotReadyError`, or
`VectorIndexNotReadyError` (all exported from `@soulcraft/brainy`), naming the
reasons. `repairIndex({ rebuild: ['metadata' | 'graph' | 'vector'] | 'all' })` is
the new explicit operator door: it rebuilds the named family unconditionally, no
health check consulted — reach for it when you have independent reason to
distrust a family regardless of what it self-reports. Bare `repairIndex()` is
unchanged in spirit: report-driven, heals only what its own checks say needs it.
- New concept doc: [Index Health](docs/concepts/index-health.md) walks the whole
story from a consumer's side — degraded-but-serving vs. not-ready, what
`repairIndex()` checks and heals per family, what `suspect` counts mean.
**Nothing to change to adopt this.** No API removed, no signature narrowed —
`repairIndex()` gains an optional options bag and its return value gains fields,
both additive. The honest notes: if your code ever relied on a `find()` against a
cold/not-yet-built index quietly triggering a rebuild and returning results a beat
later, that behavior is gone — it now throws one of the three typed
`*NotReadyError` classes instead (catch them if you need to distinguish "not ready
yet" from "no results"). And `disableAutoRebuild: true` no longer defers index
construction to the first query — a needed rebuild always runs at `open()` now;
the flag has no effect on timing. Full manual control still lives in
`repairIndex({ rebuild: [...] })`.
- **Crash-reopen catchup.** After an unclean shutdown, the metadata index now
folds the exact fact window it missed — `find()` serves every acked write on
reopen, closing the gap where canonical reads and counts recovered a
crash-window write but the index kept serving its pre-crash state until the
next full rebuild. Related root-cause fixed alongside: `close()` never
stamped the index watermarks (only `flush()` did), so a close without a
prior flush caused a needless full rescan verdict on the next open.
- **Relation rows are live in the metadata index.** Previously verb rows
entered the metadata index only during a rebuild — so a rebuilt store's
relation postings went stale from the first `relate()` after it. Relations
are now posted and retracted on the live write path (relate / unrelate /
updateRelation / remove's cascade, and their `transact()` forms), in the
same commit as the graph leg.
- **The metadata rebuild is online.** `rebuild()` for the metadata family no
longer clears and rebuilds in place (reads went empty for the duration): it
builds a complete replacement beside the serving index, mirrors concurrent
writes to both, swaps atomically, and persists once after the swap. Reads
never observe a partial index. `repairIndex({ rebuild: ['metadata'] })` uses
it automatically.
- **A broken accelerator install can never read as "not installed."** The
auto-detection free pass now requires the resolution error to name the
accelerator package itself, exactly — a missing platform-binary sibling
package, an inner file path, or a dependency failure is a broken install and
`init()` throws loudly. And a plugin that declines activation is narrated on
the always-on log channel, so `silent: true` can no longer hide a fallback
to the default engines.
---
## v10.3.1 — 2026-08-18 (the fold that behaves) ## v10.3.1 — 2026-08-18 (the fold that behaves)
Three recovery cures from one production first-boot incident (a brain's first Three recovery cures from one production first-boot incident (a brain's first

View file

@ -323,58 +323,24 @@ Only the graph adjacency index carries a committed scale assertion:
- ✅ **Single-Node by Design**: One process owns one `path`; scale out at the service layer - ✅ **Single-Node by Design**: One process owns one `path`; scale out at the service layer
- ✅ **Zero Stubs**: Every line of code is production-ready - ✅ **Zero Stubs**: Every line of code is production-ready
## Lazy Loading Performance ## Index Build at Open (10.4+)
Brainy supports two initialization modes for optimal performance across different use cases: As of 10.4, `brain.init()` runs every needed index rebuild to completion before
it returns — always, regardless of dataset size. There is no lazy,
first-query rebuild path: a brain either finishes opening healthy, or `init()`
fails loudly. `disableAutoRebuild` no longer defers index construction to a
first query; it has no effect on *when* a rebuild runs. Manual control over
rebuilds is `repairIndex({ rebuild: [...] })`. See
[Index Health](concepts/index-health.md) for the full read-gate contract
(providers self-report readiness via `healthReport()`; a read against a
not-serving provider throws a typed `*NotReadyError` rather than rebuilding
mid-query).
### Mode 1: Auto-Rebuild (Default) <!-- The pre-10.4 "Mode 2: Lazy Loading on First Query" section previously
documented here (disableAutoRebuild deferring index construction to the
```javascript first find() call) described a real, now-retired code path. Removed
const brain = new Brainy() rather than left to mislead; the concept doc above is the current
await brain.init() // Rebuilds indexes during init (~500ms-3s for 10K entities) contract. -->
```
**Performance:**
- Init time: 500ms-3s (depends on dataset size)
- First query: Instant (indexes already loaded)
- Use case: Traditional applications, long-running servers
### Mode 2: Lazy Loading
```javascript
const brain = new Brainy({ disableAutoRebuild: true })
await brain.init() // Returns instantly (0-10ms)
const results = await brain.find({ limit: 10 }) // First query triggers rebuild (~50-200ms)
const more = await brain.find({ limit: 100 }) // Subsequent queries instant (0ms check)
```
**Performance:**
- Init time: 0-10ms (instant)
- First query: 50-200ms (includes index rebuild for 1K-10K entities)
- Subsequent queries: 0ms check (instant)
- Concurrent queries: Wait for same rebuild (mutex prevents duplicates)
**Concurrency Safety:**
```javascript
// 100 concurrent queries immediately after init
await brain.init()
const promises = Array.from({ length: 100 }, () =>
brain.find({ limit: 10 })
)
const results = await Promise.all(promises)
// ✅ Only 1 rebuild triggered (mutex)
// ✅ All 100 queries return correct results
// ✅ Total time: ~60ms (not 6000ms!)
```
**Use Cases for Lazy Loading:**
- **Serverless/Edge**: Minimize cold start time (0-10ms init)
- **Development**: Faster restarts during development
- **Large datasets**: Defer index loading until needed
- **Read-heavy workloads**: Writes don't wait for index rebuild
## Zero Configuration Required ## Zero Configuration Required
@ -384,10 +350,6 @@ Brainy is designed to be **smart enough to tune itself dynamically**. No configu
// That's it. Brainy handles everything. // That's it. Brainy handles everything.
const brain = new Brainy() const brain = new Brainy()
await brain.init() await brain.init()
// Or with lazy loading for serverless
const brain = new Brainy({ disableAutoRebuild: true })
await brain.init() // Instant (0-10ms)
``` ```
### Automatic Self-Tuning ### Automatic Self-Tuning
@ -395,7 +357,6 @@ await brain.init() // Instant (0-10ms)
- **Metadata Index**: Auto-builds sorted indices for range queries on first use - **Metadata Index**: Auto-builds sorted indices for range queries on first use
- **Graph Index**: Auto-flushes every 30 seconds - **Graph Index**: Auto-flushes every 30 seconds
- **Default Tuning**: Research-based vector index defaults - **Default Tuning**: Research-based vector index defaults
- **Lazy Loading**: Indices built only when needed
- **Cache Management**: LRU caches with TTL - **Cache Management**: LRU caches with TTL
### Intelligent Defaults ### Intelligent Defaults

View file

@ -10,7 +10,7 @@ next:
- guides/storage-adapters - guides/storage-adapters
--- ---
# Plugin Development Guide # Plugin System
Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cor` provides optional native acceleration, and it's the same system available to any developer. Brainy has a plugin system that allows third-party packages to replace internal subsystems with custom implementations. This is how `@soulcraft/cor` provides optional native acceleration, and it's the same system available to any developer.
@ -200,15 +200,30 @@ members so a warm reopen never pays a redundant rebuild-from-canonical:
- **`init?(): Promise<void>`** — eager cold-load. Brainy awaits it once during - **`init?(): Promise<void>`** — eager cold-load. Brainy awaits it once during
`brain.init()`, after the metadata provider's `init()` (the id-mapper hydrates first) `brain.init()`, after the metadata provider's `init()` (the id-mapper hydrates first)
and **before the rebuild gate**. and **before the rebuild gate**.
- **`isReady?(): boolean`** — honest durability signal. `true` ⇔ the persisted index is - **`healthReport?(): HealthReport`** — the PREFERRED signal (10.4+). A named,
loaded (or cheaply demand-loadable) and consistent with what was last persisted. When synchronous, O(1) verdict derived from the provider's own exact ledgers — never a
exposed, the rebuild gate defers to this signal **instead of** the `size() === 0` / sample, never I/O, must never throw for a well-formed provider. Brainy's read gate
`totalEntries === 0` heuristics — a disk-native index may report 0 resident entries (`assessProviderHealth()`) reads this INSTEAD of `isReady()` / size heuristics when
while fully durable. Never return `true` if the durable state failed to load: the present: `serving: false` refuses the read with a typed `*NotReadyError` rather than
signal is honest in both directions, and a not-ready provider gets its rebuild even triggering a rebuild — a read never starts a store walk. `healthy` marks every
when `size() > 0`. *verified* invariant holding; a family named in `unledgered` counts as neither
healthy nor broken. See `HealthReport` / `LedgerInvariantResult` /
`InvariantSource` in `src/plugin.ts`, and
[Index Health](concepts/index-health.md) for the consumer-facing story.
- **`isReady?(): boolean`** — honest durability signal, the fallback when
`healthReport()` is absent. `true` ⇔ the persisted index is loaded (or cheaply
demand-loadable) and consistent with what was last persisted. When exposed, the
gate defers to this signal **instead of** the `size() === 0` / `totalEntries === 0`
heuristics — a disk-native index may report 0 resident entries while fully durable.
Never return `true` if the durable state failed to load: the signal is honest in
both directions, and a not-ready provider gets its rebuild even when `size() > 0`.
- **`isMigrating?(): boolean`** — while `true`, the provider owns its index (background - **`isMigrating?(): boolean`** — while `true`, the provider owns its index (background
migration); brainy skips its rebuild entirely. migration); brainy skips its rebuild entirely.
- **`validateInvariants?(): Promise<ProviderInvariantReport>`** — the async DEEP
diagnostic (full scans allowed), distinct from the bounded, sync `healthReport()`.
Must never throw — a failure is `healthy: false` data, not an exception; a provider
that throws anyway is read as a loud, unverified failure (never as "healthy") by
every caller, never silently retried into a rebuild.
Providers that implement none of these keep the size/count heuristics — correct for Providers that implement none of these keep the size/count heuristics — correct for
engines whose `rebuild()` *is* their load path (like brainy's built-in JS vector index). engines whose `rebuild()` *is* their load path (like brainy's built-in JS vector index).

View file

@ -1451,6 +1451,34 @@ const count = await brain.getVerbCount()
--- ---
### The canonical count ledger (`StorageAdapter.getCanonicalCounts()`)
An OPTIONAL method on the `StorageAdapter` interface (implemented by both
built-in adapters), not a method on `Brainy` itself — relevant if you're
writing a custom storage adapter or composing a provider's own
`healthReport()`. O(1), no I/O. Per family (`nouns`/`verbs`):
```typescript
interface CanonicalCounts {
nouns: { counted: number; all: number }
verbs: { counted: number; all: number }
suspect: boolean
}
```
- `counted` mirrors `getNounCount()` / `getVerbCount()` (public + internal tiers).
- `all` is the ALL-visibility scalar — every tier, including system/internal
records — the denominator a derived index's own coverage math is measured
against.
- `suspect` is `true` when an unprovable delete has left `all` unverified since
the last recount; `brain.repairIndex()` clears it with a real canonical walk.
Adapters without the ledger omit the method; treat absence as "no
denominator," never as zero. See
**[Index Health](../concepts/index-health.md)** for the full story.
---
### Subtype & facet APIs ### Subtype & facet APIs
Full guide: **[Subtypes & Facets](../guides/subtypes-and-facets.md)**. Full guide: **[Subtypes & Facets](../guides/subtypes-and-facets.md)**.
@ -1852,6 +1880,81 @@ await brain.repairIndex({ rebuild: ['graph'] })
await brain.repairIndex({ rebuild: 'all' }) await brain.repairIndex({ rebuild: 'all' })
``` ```
**`RepairReport`:**
- `families: RepairFamilyReport[]` — one row per family checked
- `healedTotal: number` — items healed across every family
- `durationMs: number`
**`RepairFamilyReport`** (one row):
- `family: string` — e.g. `'orphaned-containers'`, `'count-rollups'`,
`'vfs-containment'`, `'metadata-corruption'`, `'provider:metadata'`,
`'provider:graph'`, `'provider:vector'`
- `checked: boolean` — was this family actually examined (`false` ⇒ see `skipped`)
- `healed: number` — items re-posted/corrected in place (the incremental heal count)
- `missing?: { count: number; sample: string[] }` — exact count plus a capped id
sample when the check can name what diverged (never the full list)
- `rebuilt?: boolean` — a full generational rebuild ran (vs. an incremental heal)
- `detail?: string` / `reason?: string` — narration
- `skipped?: string` — why the family wasn't checked
Full walkthrough — what each family checks, degraded-but-serving vs. not-ready,
and what `suspect` counts mean — in
**[Index Health](../concepts/index-health.md)**.
---
### Index readiness: typed errors, `healthReport()`, `disableAutoRebuild`
Every derived-index provider (vector, graph, metadata) may expose a named,
synchronous, O(1) `healthReport()` composed from its own exact ledgers — the
signal Brainy's read gate trusts over sampling or size heuristics. `init()`
brings every provider to serving before it returns; there is no first-query
lazy-rebuild path. A read that reaches a provider whose health report says it
isn't serving throws instead of rebuilding mid-query:
| Error | Thrown by | Meaning |
|---|---|---|
| `GraphIndexNotReadyError` | `find({ connected })`, `neighbors()`, `related()` | Graph adjacency isn't serving |
| `MetadataIndexNotReadyError` | `find({ where })` | Metadata/field index isn't serving |
| `VectorIndexNotReadyError` | `find({ query })`, `similar()` | Vector index isn't serving |
All three are exported from `@soulcraft/brainy`. Catch them to distinguish
"index not ready" from a genuine empty result:
```typescript
import { MetadataIndexNotReadyError } from '@soulcraft/brainy'
try {
const rows = await brain.find({ where: { status: 'active' } })
} catch (err) {
if (err instanceof MetadataIndexNotReadyError) {
// reconcile: await brain.repairIndex(), then retry
} else {
throw err
}
}
```
**`disableAutoRebuild`** no longer defers index construction to the first
query. A needed rebuild always runs at `open()`, regardless of this flag or
dataset size; the flag has no effect on *when* a rebuild runs. Full manual
control lives in `repairIndex({ rebuild: [...] })`, above.
### `validateIndexConsistency()``Promise<...>`
The deep, async diagnostic counterpart to `healthReport()` — safe to run on a
live brain, but does more work (a provider's `validateInvariants()` may run a
full scan, not just read a ledger). Aggregates the JS metadata index's own
consistency check with every derived-index provider's invariant report.
```typescript
const validation = await brain.validateIndexConsistency()
if (!validation.healthy) {
console.log(validation.recommendation) // what to run, e.g. repairIndex()
console.log(validation.providers) // each provider's own invariant report, when exposed
}
```
--- ---
## Lifecycle ## Lifecycle

View file

@ -723,6 +723,14 @@ async stats(): Promise<Statistics> {
### 5. Index Rebuilding (Lazy Loading Support) ### 5. Index Rebuilding (Lazy Loading Support)
> **Stale as of 10.4 — "Mode 2: Lazy Loading on First Query" below is
> RETIRED.** `disableAutoRebuild` no longer defers index construction to a
> first query; `brain.init()` now runs every needed rebuild to completion
> before it returns, unconditionally, and a read against a not-serving
> provider throws a typed `*NotReadyError` instead of rebuilding mid-query.
> See `docs/concepts/index-health.md` for the current contract. Left below
> as historical background on the rebuild mechanics.
**Two modes of index loading:** **Two modes of index loading:**
#### Mode 1: Auto-Rebuild on init() (default) #### Mode 1: Auto-Rebuild on init() (default)

View file

@ -1,5 +1,15 @@
# Initialization and Rebuild Processes # Initialization and Rebuild Processes
> **Stale as of 10.4 — "Mode 2: Lazy Loading on First Query" below is RETIRED.**
> `disableAutoRebuild` no longer defers index construction to a first query;
> `brain.init()` now runs every needed rebuild to completion before it
> returns, unconditionally. A read against a not-serving provider throws a
> typed `*NotReadyError` instead of rebuilding mid-query. See
> `docs/concepts/index-health.md` for the current contract; this document's
> line-number references to `src/brainy.ts` also predate the file's current
> size and are unreliable. Left as historical background on the rebuild
> mechanics, not as a current API description.
This document explains how Brainy's four indexes (MetadataIndex, vector index, GraphAdjacencyIndex, DeletedItemsIndex) initialize and rebuild from persisted storage. This document explains how Brainy's four indexes (MetadataIndex, vector index, GraphAdjacencyIndex, DeletedItemsIndex) initialize and rebuild from persisted storage.
## Core Principle: All Indexes Are Disk-Based ## Core Principle: All Indexes Are Disk-Based

View file

@ -0,0 +1,204 @@
---
title: Index Health
slug: concepts/index-health
public: true
category: concepts
template: concept
order: 8
description: How Brainy knows whether a derived index can be trusted — exact accounting instead of sampling, the named health report, degraded-but-serving vs. not-ready, and what repairIndex() checks, heals, and rebuilds.
next:
- concepts/generation-fact-log
- guides/inspection
---
# Index Health
Brainy keeps one **canonical** copy of every entity and relationship, and three
**derived** indexes built from it — vector, metadata, and graph — so `find()` can
answer semantically, by filter, and by traversal without re-deriving the answer from
scratch on every query. A derived index is a cache with a serving structure: it can
be present but stale, present but only partially loaded, or fully out of sync with
canonical after a crash. This page is about how Brainy decides whether to trust one,
what it does when it can't, and how you reconcile the two.
## Exact accounting instead of sampling
Older health checks worked by inference: does `size()` return something greater
than zero, does a spot-check on one known item come back correct. Both are proxies.
A cold index can report a nonzero count while its actual serving structure never
loaded, and a spot-check only proves the one item it happened to ask about.
Every derived-index provider may now expose a named, synchronous, O(1)
`healthReport()` — composed from the provider's own **exact ledgers** (real counters
it already maintains on the write path), never a sample or a walk. This is the one
signal Brainy's read gate consults. A provider that doesn't yet expose one falls
back to an honest `isReady()` boolean, and finally to a size heuristic for engines
with neither — but wherever a `healthReport()` exists, it wins.
Underneath, storage itself keeps an analogous **canonical count ledger**: a
`counted` scalar (the user-facing total — what `getNounCount()` / `getVerbCount()`
return) and an `all` scalar (every tier, including internal records a derived
index's own coverage math needs to compare against). This is the real denominator
a provider's `healthReport()` measures itself by, rather than a total that can only
ever ratchet upward. See [What `suspect` counts mean](#what-suspect-counts-mean)
below for the one case that ledger can't stay exact through on its own.
## The named report
A `HealthReport` carries, per provider (`'vector'` / `'graph'` / `'metadata'`):
- **`healthy`** — `true` iff every *verified* invariant holds. An invariant whose
family has no ledger yet is `unledgered`, never counted either way — unknown,
not passing.
- **`serving`** — can this provider answer a query right now. A failing invariant
graded `heal: 'repair'` or `heal: 'none'` still leaves `serving: true` — this is
**degraded-but-serving**: something is off (say, a stale rollup on an
`employee` record's relationship count) but reads keep working. Only a failure
graded `heal: 'rebuild'` flips `serving` to `false`**not-ready** — because the
provider itself is telling you its serving structure cannot answer correctly.
- **`invariants`** — each checked condition, with its provenance
(`source: 'ledger'` — an exact count; `'deep'` — a full scan, diagnostic-only;
`'unledgered'` — not yet tracked) and, for a failing one, an exact `missing`
count plus a capped sample of the affected ids — a verdict, never a dump.
- **`generation`** — bumps on every ledger mutation and rebuild, so a caller can
cache a verdict per generation instead of re-deriving it.
The distinction that matters day to day: `healthy: false` can be entirely benign —
a maintenance window, a divergence `repairIndex()` will clean up on its own
schedule. `serving: false` is not benign. It means this provider is refusing to
answer, on its own word, right now.
## Reads refuse — they never rebuild
A query that reaches a not-serving provider does not trigger a rebuild from inside
the read. Brainy retired that path deliberately: a rebuild kicked off by an ordinary
`find({ where: { status: 'active' } })` call is a dark, unpredictable cost hiding
behind a request that looks like a cheap read. Instead, the read throws a typed,
catchable error naming the reason:
| Error | Thrown when | Meaning |
|---|---|---|
| `GraphIndexNotReadyError` | `find({ connected })`, `neighbors()`, `related()` | The graph adjacency index isn't serving — traversal would otherwise return `[]` indistinguishable from "no relationships" |
| `MetadataIndexNotReadyError` | `find({ where })` | The metadata/field index isn't serving — a filtered read would otherwise return `[]` indistinguishable from "no matches" |
| `VectorIndexNotReadyError` | `find({ query })`, `similar()` | The vector index isn't serving — a semantic search would otherwise return `[]` indistinguishable from "nothing similar" |
All three are exported from `@soulcraft/brainy`. Catch them where your application
needs to distinguish "this index isn't ready yet" from "there's genuinely nothing
here" — a health dashboard, a retry policy, an operator alert. The fix is always
the same: reconcile the index, either by reopening the brain (which brings every
provider to serving before `init()` returns — see the next section) or by calling
`repairIndex()` explicitly.
```typescript
try {
const active = await brain.find({ where: { status: 'active' } })
} catch (err) {
if (err instanceof MetadataIndexNotReadyError) {
// not a "no results" — the index itself refused; alert or retry after repair
} else {
throw err
}
}
```
### Rebuilds happen at open, not on first query
`brain.init()` runs every needed rebuild to completion **before it returns**,
unconditionally, regardless of dataset size. There is no lazy, first-query
rebuild path anymore — a brain either finishes opening healthy, or it fails
open loudly. `disableAutoRebuild: true` no longer defers index construction to
the first query: it has no effect on *when* a needed rebuild runs. Full manual
control over rebuilds is `repairIndex({ rebuild: [...] })` (below), not this flag.
## `repairIndex()` — checking and healing
Bare `repairIndex()` is **report-driven**: it only heals what its own checks say
actually needs it, and it always returns a full per-family receipt.
```typescript
const report = await brain.repairIndex()
report.healedTotal // total items healed across every family
report.durationMs
report.families // one row per family checked
```
Each `RepairFamilyReport` row names what happened:
- **`checked`** — was this family actually examined (`false` means skipped —
see `skipped` for why).
- **`healed`** — items re-posted or corrected in place.
- **`missing`** — when the check can name what diverged: an exact `count` plus a
capped `sample` of ids.
- **`rebuilt`** — a full generational rebuild ran (as opposed to an incremental
heal).
- **`detail`** / **`reason`** / **`skipped`** — the receipt's narration; a row is
always either checked or explains why it wasn't. Nothing is silent.
On every call, bare `repairIndex()`:
1. Prunes orphaned canonical containers left by a partial delete.
2. Recomputes the count rollups from one canonical walk (unconditional — this is
also what clears a `suspect` ledger; see below).
3. Reconciles VFS containment edges, if the VFS is initialized.
4. Runs the metadata index's own corruption detection pass.
5. Consults each of the three derived-index providers' own health check and
rebuilds only a family whose failing invariant actually asks for it
(`heal: 'rebuild'`) — never a provider that reports `healthy` or a lesser
grade.
### The explicit rebuild door
`options.rebuild` skips the health check and rebuilds one or more families
**unconditionally** — the operator override for when you have independent reason
to distrust a family regardless of what it self-reports (a suspicious deploy, a
storage-layer incident, a support ticket that doesn't match what the health report
says):
```typescript
// Force the graph adjacency to rebuild from canonical, no invariant consulted
await brain.repairIndex({ rebuild: ['graph'] })
// Force all three derived indexes
await brain.repairIndex({ rebuild: 'all' })
```
A family named this way is recorded with `rebuilt: true` and
`reason: 'explicit rebuild requested'`, and is skipped by the normal
health-driven pass in the same call — it was already rebuilt unconditionally.
Reach for the explicit door when you need certainty regardless of self-report;
reach for bare `repairIndex()` for routine maintenance and after any incident
where you're not sure which family (if any) needs it.
## What `suspect` counts mean
Storage's canonical count ledger increments the ALL-visibility total on every new
record and decrements it on every *proven* delete — one where the record was read,
or the caller supplied its prior image. A delete that cannot prove what it removed
existed doesn't guess: it flags the ledger `suspect` (an operator-visible
`console.warn`, narrated once per session, not once per delete) rather than risk
decrementing a total that was never incremented for that record in the first
place. This is intentionally rare — it's a defensive fallback for callers on an
unusual removal path, not a per-delete cost.
`suspect` is not directly exposed on any `Brainy` method today — it lives on the
`StorageAdapter`'s optional `getCanonicalCounts()`, primarily consulted by
`repairIndex()`'s recount step and by custom storage adapters composing their own
`healthReport()`. What matters for an application: a `suspect` ledger is not
incorrect, just *unverified since the last recount* — and `repairIndex()`'s
unconditional count-rollup step (step 2, above) recomputes the ALL scalars from a
real canonical walk on every call, clearing the flag with proof either way.
## Practical guidance
- **On a normal restart**, do nothing — `init()` brings every provider to
serving before it returns, or fails loudly.
- **On a `*NotReadyError`** from a live read, reconcile with `repairIndex()`
(report-driven is almost always sufficient) and retry.
- **After an incident** where you distrust a specific family regardless of what
it reports healthy — a storage-layer fault, a suspicious restore — use the
explicit door: `repairIndex({ rebuild: ['metadata' | 'graph' | 'vector'] })`.
- **To audit before trusting a report**, `brain.auditGraph()` walks every stored
relationship and proves (or disproves) that reads return canonical truth,
independent of what any provider self-reports — see
[Inspecting a Live Brainy](../guides/inspection.md).

View file

@ -14,3 +14,9 @@ from the message alone. `biography.test.ts` is split into two `it` blocks
(Ch1-3, then Ch4-6) purely for reporting; it is still ONE fixed-order story. (Ch1-3, then Ch4-6) purely for reporting; it is still ONE fixed-order story.
Chapters must never be reordered, skipped, or made conditional, and a Chapters must never be reordered, skipped, or made conditional, and a
failing chapter's assertion must never be weakened to force green. failing chapter's assertion must never be weakened to force green.
Lab notes (hard-won, keep):
- `git reset --hard` does NOT remove untracked files — a "clean" tree can still
carry stray test stores; use `git clean -fd tests/lifecycle-tmp` equivalents.
- `silent: true` patches `console` process-wide — never assert narration through
`console` spies in this lane; the engine's always-on channel is `prodLog`.