brainy/RELEASES.md

@soulcraft/brainy — Release Notes for Consumers

This file is the quick reference for Soulcraft product sessions tracking Brainy changes. Full auto-generated changelog: CHANGELOG.md · Releases: https://github.com/soulcraftlabs/brainy/releases

How to use: Brainy is the underlying data engine for Workshop, Venue, Academy, and Collective. The SDK wraps it — most products never call Brainy directly. Read this when:

• Upgrading @soulcraft/brainy in the SDK or a product
• Debugging data, query, or storage behaviour
• A new Brainy feature is available that SDK should expose

---

v7.21.0 — 2026-05-15

Affected products: All consumers using filesystem storage (Venue, Workshop dev, local development).

Multi-process safety + first-class read-only inspector

Filesystem storage now enforces single-writer, many-reader. Previously, opening a second writer process against a live data directory silently produced wrong query results — no error, no warning, just empty or stale data. This release replaces that footgun with a hard refusal at init time and a dedicated read-only inspection mode.

New: Brainy.openReadOnly()

const reader = await Brainy.openReadOnly({
  storage: { type: 'filesystem', rootDirectory: '/data/brain' }
})
const bookings = await reader.find({ where: { entityType: 'booking' } })

• Does NOT acquire the writer lock — coexists with a live writer.
• Every mutation method throws Cannot mutate a read-only Brainy instance.
• flush() and close() are safe (no-op + clean shutdown).

New: writer lock at init

• new Brainy({ ... }) (default mode: 'writer') acquires <rootDir>/locks/_writer.lock containing { pid, hostname, startedAt, lastHeartbeat, version }.
• A second writer on the same directory throws with the PID, hostname, heartbeat, and a pointer to openReadOnly().
• Stale-lock detection: same hostname + dead PID OR heartbeat > 60s ago → overwritten with a warning.
• Heartbeat: writer rewrites lastHeartbeat every 10s (unref'd timer).
• Override: pass { force: true } to bypass when you've verified the existing lock is stale.

New: brain.requestFlush({ timeoutMs })

Cross-process RPC for inspectors to force a fresh snapshot:

• In-process: just calls flush().
• Out-of-process: writes a request file, polls for ack. Times out gracefully.
• Watcher runs in every writer instance (filesystem only).

New: brain.stats()

Operator-facing summary: entityCount, entitiesByType, relationCount, relationsByType, fieldRegistry, indexHealth, storage.backend, writerLock, version. Designed for /api/health endpoints and incident triage.

New: brain.explain(findParams)

Shows which index path serves each where clause: column-store | sparse-chunked | none. This is the answer to "why is find() returning empty?" — path: "none" means the field has no index entries and the query will silently return []. Includes notes on likely causes (writer hasn't flushed; typo; field genuinely absent).

New: brain.health()

Invariant-check battery: HNSW vs metadata count parity, field registry sanity, _seeded entity sweep, writer heartbeat freshness. Returns { overall: 'pass' | 'warn' | 'fail', checks: [...] }.

New: brainy inspect CLI (13 subcommands)

All read-only by default (internally uses openReadOnly()):

brainy inspect stats   <path>
brainy inspect find    <path> --type Event --where '{"status":"paid"}' --limit 20
brainy inspect get     <path> <id>
brainy inspect relations <path> <id> --direction both
brainy inspect explain <path> --where '{"entityType":"booking"}'
brainy inspect health  <path>
brainy inspect sample  <path> --type Event --n 20
brainy inspect fields  <path>
brainy inspect dump    <path> --type Event > backup.jsonl
brainy inspect watch   <path> --type Event
brainy inspect backup  <path> /backups/brain.tar
brainy inspect repair  <path>           # writer mode — stop live writer first
brainy inspect diff    <pathA> <pathB>

Default behaviour: ask the writer to flush via the RPC first (skip with --no-fresh).

Brainy + Cortex

Cortex segments (*.cidx) are immutable mmap files; MANIFEST.json uses atomic rename. The single writer lock at <rootDir>/locks/_writer.lock covers both Brainy and Cortex. Read-only inspectors mmap Cortex segments with zero coordination.

What's NOT enforced yet

• Cloud storage backends (S3, GCS, R2, Azure) — no multi-process locking. A best-effort warning logs in writer mode against non-filesystem backends.
• The find({ where })-returns-0 root-cause bug — tracked separately. The new brain.explain() and brain.health() surface it loudly (path: 'none', index-parity warn) instead of letting it be silent.

Upgrade notes

• Default behaviour change: opening a Brainy directory in writer mode while another writer is live now THROWS where it previously silently produced wrong results. If you have scripts that intentionally co-opened a writer directory, switch them to Brainy.openReadOnly() or pass { force: true }.
• Cloud backends unchanged — no lock acquisition for S3/GCS/R2/Azure.
• All existing APIs unchanged. Pure addition.

Docs

• README "Single-Writer Model" section.
• docs/concepts/multi-process.md — full model + Cortex compatibility.
• docs/guides/inspection.md — operator recipes.
• JSDoc on every new API.

---

v7.19.10 — 2026-02-24

Affected products: All Bun/ESM consumers (Workshop, Venue, Academy, SDK)

ESM crypto fix in SSTable

Replaced require('crypto') with import { createHash } from 'node:crypto' in the SSTable implementation. Fixes a crash in Bun and strict ESM environments where CommonJS require is unavailable.

No API changes — upgrade and redeploy.

---

v7.19.2 — 2026-02-18

Affected products: All

Metadata index cleanup on delete

Fixed: metadata indexes were not cleaned up after delete() / deleteMany(). Stale index entries could cause phantom results in metadata-filtered queries after deletion.

No API changes. If you were seeing ghost results in filtered queries, this fixes it.

---

v7.18.0 — 2026-02-16

Affected products: Workshop, Venue, Academy (analytics, reporting, session summaries)

Aggregation engine

New brain.aggregate() API — incremental SUM, COUNT, AVG, MIN, MAX with GROUP BY and time window support. Computes over entity collections without loading all records into memory.

const result = await brain.aggregate({
  collection: 'bookings',
  metrics: [
    { field: 'revenue', fn: 'SUM' },
    { field: 'id', fn: 'COUNT' },
  ],
  groupBy: 'staffId',
  timeWindow: { field: 'createdAt', from: startOfMonth, to: now },
})

SDK exposure: sdk.brainy.aggregate() — available once SDK is updated to pass through.

---

v7.17.0 — 2026-02-09

Affected products: All (schema evolution, data migrations)

Migration system

New brain.migrate() API with error handling, validation, and enterprise hardening. Run schema migrations reliably across Brainy data directories.

await brain.migrate({
  version: 3,
  up: async (brain) => {
    // transform entities, rename fields, etc.
  },
})

---

v7.16.0 — 2026-02-09

Affected products: All

Data/metadata separation enforced + numeric range queries

• Entity data and metadata fields are now strictly separated at the storage layer
• Numeric range queries now supported in metadata filters: { age: { $gte: 18, $lt: 65 } }
• Fixes edge cases where mixed data/metadata storage caused inconsistent query results

Breaking for anyone storing numeric values in metadata and relying on range queries: verify your filter syntax matches the new $gte/$lte/$gt/$lt operators.

---

v7.15.5 — 2026-02-02

Affected products: Anyone using @soulcraft/cortex plugin

Plugin opt-in clarified

Cortex and other plugins are opt-in. Pass explicitly:

new Brainy({ plugins: ['@soulcraft/cortex'] })

Without plugins, no external plugins are loaded regardless of what's installed.

---

v7.15.2 — 2026-02-01

Affected products: All (data safety)

Graph LSM flush on close

Fixed: graph LSM-trees were not flushed on brain.close(), risking data loss across restarts. Graph edges written in the final seconds before shutdown are now guaranteed to be persisted.

No API changes — upgrade immediately if running Brainy in a long-lived server process.