feat: multi-process safety + read-only inspector mode
Filesystem storage now enforces single-writer, many-reader semantics. A second writer on the same data directory throws at init time with the holder's PID, hostname, and heartbeat — replacing the previous silent stale-reads failure mode. - New: `Brainy.openReadOnly()` — coexists with a live writer, every mutation throws clearly. - New: writer lock at `<rootDir>/locks/_writer.lock` with 10s heartbeat and stale-detection (PID liveness + heartbeat freshness). - New: cross-process flush-request RPC (filesystem-based, no signals) so inspectors can force fresh state on demand. - New: `brain.stats()`, `brain.explain(findParams)`, `brain.health()` for operator-facing introspection. - New: `brainy inspect` CLI with 13 subcommands (stats, find, get, relations, explain, health, sample, fields, dump, watch, backup, repair, diff), all read-only by default. - Same-PID re-opens allowed with a warning (preserves test "simulate restart" patterns). - Storage instances passed directly via `storage: new MemoryStorage()` are now honoured instead of silently falling through to the filesystem auto-detect path. Brainy + Cortex compose under this model — the lock covers both because they share `rootDir`, Cortex segments are immutable mmap files, and MANIFEST updates use atomic-rename.
This commit is contained in:
parent
1bc6a430c7
commit
4fcdc0fef3
12 changed files with 2342 additions and 7 deletions
105
RELEASES.md
105
RELEASES.md
|
|
@ -11,6 +11,111 @@ Collective. The SDK wraps it — most products never call Brainy directly. Read
|
|||
|
||||
---
|
||||
|
||||
## v7.21.0 — 2026-05-15
|
||||
|
||||
**Affected products:** All consumers using filesystem storage (Venue, Workshop dev,
|
||||
local development).
|
||||
|
||||
### Multi-process safety + first-class read-only inspector
|
||||
|
||||
Filesystem storage now enforces **single-writer, many-reader**. Previously, opening
|
||||
a second writer process against a live data directory silently produced wrong query
|
||||
results — no error, no warning, just empty or stale data. This release replaces
|
||||
that footgun with a hard refusal at init time and a dedicated read-only inspection
|
||||
mode.
|
||||
|
||||
#### New: `Brainy.openReadOnly()`
|
||||
```ts
|
||||
const reader = await Brainy.openReadOnly({
|
||||
storage: { type: 'filesystem', rootDirectory: '/data/brain' }
|
||||
})
|
||||
const bookings = await reader.find({ where: { entityType: 'booking' } })
|
||||
```
|
||||
- Does NOT acquire the writer lock — coexists with a live writer.
|
||||
- Every mutation method throws `Cannot mutate a read-only Brainy instance`.
|
||||
- `flush()` and `close()` are safe (no-op + clean shutdown).
|
||||
|
||||
#### New: writer lock at init
|
||||
- `new Brainy({ ... })` (default `mode: 'writer'`) acquires
|
||||
`<rootDir>/locks/_writer.lock` containing `{ pid, hostname, startedAt, lastHeartbeat, version }`.
|
||||
- A second writer on the same directory **throws** with the PID, hostname,
|
||||
heartbeat, and a pointer to `openReadOnly()`.
|
||||
- Stale-lock detection: same hostname + dead PID OR heartbeat > 60s ago → overwritten with a warning.
|
||||
- Heartbeat: writer rewrites `lastHeartbeat` every 10s (unref'd timer).
|
||||
- Override: pass `{ force: true }` to bypass when you've verified the existing lock is stale.
|
||||
|
||||
#### New: `brain.requestFlush({ timeoutMs })`
|
||||
Cross-process RPC for inspectors to force a fresh snapshot:
|
||||
- In-process: just calls `flush()`.
|
||||
- Out-of-process: writes a request file, polls for ack. Times out gracefully.
|
||||
- Watcher runs in every writer instance (filesystem only).
|
||||
|
||||
#### New: `brain.stats()`
|
||||
Operator-facing summary: `entityCount`, `entitiesByType`, `relationCount`,
|
||||
`relationsByType`, `fieldRegistry`, `indexHealth`, `storage.backend`, `writerLock`, `version`.
|
||||
Designed for `/api/health` endpoints and incident triage.
|
||||
|
||||
#### New: `brain.explain(findParams)`
|
||||
Shows which index path serves each `where` clause: `column-store` | `sparse-chunked` | `none`.
|
||||
**This is the answer to "why is `find()` returning empty?"** — `path: "none"` means
|
||||
the field has no index entries and the query will silently return `[]`. Includes
|
||||
notes on likely causes (writer hasn't flushed; typo; field genuinely absent).
|
||||
|
||||
#### New: `brain.health()`
|
||||
Invariant-check battery: HNSW vs metadata count parity, field registry sanity,
|
||||
`_seeded` entity sweep, writer heartbeat freshness. Returns `{ overall: 'pass' | 'warn' | 'fail', checks: [...] }`.
|
||||
|
||||
#### New: `brainy inspect` CLI (13 subcommands)
|
||||
All read-only by default (internally uses `openReadOnly()`):
|
||||
```
|
||||
brainy inspect stats <path>
|
||||
brainy inspect find <path> --type Event --where '{"status":"paid"}' --limit 20
|
||||
brainy inspect get <path> <id>
|
||||
brainy inspect relations <path> <id> --direction both
|
||||
brainy inspect explain <path> --where '{"entityType":"booking"}'
|
||||
brainy inspect health <path>
|
||||
brainy inspect sample <path> --type Event --n 20
|
||||
brainy inspect fields <path>
|
||||
brainy inspect dump <path> --type Event > backup.jsonl
|
||||
brainy inspect watch <path> --type Event
|
||||
brainy inspect backup <path> /backups/brain.tar
|
||||
brainy inspect repair <path> # writer mode — stop live writer first
|
||||
brainy inspect diff <pathA> <pathB>
|
||||
```
|
||||
Default behaviour: ask the writer to flush via the RPC first (skip with `--no-fresh`).
|
||||
|
||||
### Brainy + Cortex
|
||||
|
||||
Cortex segments (`*.cidx`) are immutable mmap files; `MANIFEST.json` uses atomic
|
||||
rename. The single writer lock at `<rootDir>/locks/_writer.lock` covers both Brainy
|
||||
and Cortex. Read-only inspectors mmap Cortex segments with zero coordination.
|
||||
|
||||
### What's NOT enforced yet
|
||||
|
||||
- **Cloud storage backends** (S3, GCS, R2, Azure) — no multi-process locking.
|
||||
A best-effort warning logs in writer mode against non-filesystem backends.
|
||||
- **The `find({ where })`-returns-0 root-cause bug** — tracked separately. The
|
||||
new `brain.explain()` and `brain.health()` surface it loudly
|
||||
(`path: 'none'`, `index-parity warn`) instead of letting it be silent.
|
||||
|
||||
### Upgrade notes
|
||||
|
||||
- **Default behaviour change:** opening a Brainy directory in writer mode while
|
||||
another writer is live now THROWS where it previously silently produced wrong
|
||||
results. If you have scripts that intentionally co-opened a writer directory,
|
||||
switch them to `Brainy.openReadOnly()` or pass `{ force: true }`.
|
||||
- **Cloud backends unchanged** — no lock acquisition for S3/GCS/R2/Azure.
|
||||
- All existing APIs unchanged. Pure addition.
|
||||
|
||||
### Docs
|
||||
|
||||
- README "Single-Writer Model" section.
|
||||
- `docs/concepts/multi-process.md` — full model + Cortex compatibility.
|
||||
- `docs/guides/inspection.md` — operator recipes.
|
||||
- JSDoc on every new API.
|
||||
|
||||
---
|
||||
|
||||
## v7.19.10 — 2026-02-24
|
||||
|
||||
**Affected products:** All Bun/ESM consumers (Workshop, Venue, Academy, SDK)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue