feat: multi-process safety + read-only inspector mode

Filesystem storage now enforces single-writer, many-reader semantics.
A second writer on the same data directory throws at init time with the
holder's PID, hostname, and heartbeat — replacing the previous silent
stale-reads failure mode.

- New: `Brainy.openReadOnly()` — coexists with a live writer, every
  mutation throws clearly.
- New: writer lock at `<rootDir>/locks/_writer.lock` with 10s heartbeat
  and stale-detection (PID liveness + heartbeat freshness).
- New: cross-process flush-request RPC (filesystem-based, no signals)
  so inspectors can force fresh state on demand.
- New: `brain.stats()`, `brain.explain(findParams)`, `brain.health()`
  for operator-facing introspection.
- New: `brainy inspect` CLI with 13 subcommands (stats, find, get,
  relations, explain, health, sample, fields, dump, watch, backup,
  repair, diff), all read-only by default.
- Same-PID re-opens allowed with a warning (preserves test "simulate
  restart" patterns).
- Storage instances passed directly via `storage: new MemoryStorage()`
  are now honoured instead of silently falling through to the
  filesystem auto-detect path.

Brainy + Cortex compose under this model — the lock covers both because
they share `rootDir`, Cortex segments are immutable mmap files, and
MANIFEST updates use atomic-rename.
This commit is contained in:
David Snelling 2026-05-15 11:25:05 -07:00
parent 1bc6a430c7
commit 4fcdc0fef3
12 changed files with 2342 additions and 7 deletions

View file

@ -80,6 +80,20 @@ export const VERBS_METADATA_DIR = 'entities/verbs/metadata'
export const SYSTEM_DIR = '_system'
export const STATISTICS_KEY = 'statistics'
/**
* Metadata persisted in the writer lock file. Used by stale-lock detection
* (PID liveness + hostname + heartbeat freshness) and by `brain.stats()` for
* operator-facing diagnostics.
*/
export interface WriterLockInfo {
pid: number
hostname: string
startedAt: string // ISO timestamp when the lock was first acquired
lastHeartbeat: string // ISO timestamp of the most recent heartbeat update
version: string // Brainy version that wrote the lock
rootDir?: string // Convenience for log lines / error messages
}
/**
* FNV-1a hash returning a 2-char hex bucket (00-ff).
* Distributes system keys across 256 sub-prefixes to avoid
@ -398,6 +412,79 @@ export abstract class BaseStorage extends BaseStorageAdapter {
}
}
/**
* Whether this storage adapter enforces multi-process writer exclusion.
* Filesystem storage returns true; cloud and memory adapters return false.
* Brainy.init() checks this to decide whether to log a multi-process warning
* in writer mode.
*/
public supportsMultiProcessLocking(): boolean {
return false
}
/**
* Attempt to acquire the process-level writer lock at init time. Default
* implementation is a no-op (multi-process safety not enforced). Filesystem
* storage overrides this with real locking semantics.
*
* @param options.force - If true, overwrite any existing writer lock. Use
* only when stale detection cannot prove the existing lock is dead.
* @returns Metadata about the acquired lock (or `null` if no lock was needed).
* @throws If another live writer holds the lock and `force` is not set.
*/
public async acquireWriterLock(options?: { force?: boolean }): Promise<WriterLockInfo | null> {
return null
}
/**
* Release the writer lock acquired by `acquireWriterLock()`. No-op if no lock
* was held. Filesystem storage overrides this to delete the lock file and
* stop the heartbeat timer.
*/
public async releaseWriterLock(): Promise<void> {
// No-op by default
}
/**
* Read the current writer-lock metadata if one is held by any process. Used
* by `brain.stats()` for diagnostics. Default returns null (no lock model).
*/
public async readWriterLock(): Promise<WriterLockInfo | null> {
return null
}
/**
* Start watching for cross-process flush requests. The writer Brainy
* instance calls this so that out-of-process inspectors can ask for a
* synchronous flush before they open the store read-only. Default is a
* no-op (non-filesystem backends have no shared filesystem to poll).
*
* @param onRequest - Callback invoked when a request file appears. Should
* call `brain.flush()` and resolve when persistence is complete.
*/
public startFlushRequestWatcher(onRequest: () => Promise<void>): void {
// No-op by default
}
/**
* Stop the flush-request watcher started by `startFlushRequestWatcher`.
*/
public stopFlushRequestWatcher(): void {
// No-op by default
}
/**
* Write a flush-request file and wait for the writer to acknowledge by
* writing the corresponding response file. Returns true if a response was
* received before the timeout, false if it timed out.
*
* Cross-platform RPC over the shared filesystem no signals, so this works
* on Windows, Linux, macOS, and inside containers without IPC config.
*/
public async requestFlushOverFilesystem(_timeoutMs: number): Promise<boolean> {
return false
}
/**
* Lightweight COW enablement - just enables branch-scoped paths
* Called during init() to ensure all data is stored with branch prefixes from the start