David Snelling
38b0041464
Every committed generation now also appends a FACT — an after-image commit record (what each touched entity/relationship became, or a body-less tombstone for a removal) — to an append-only, crc32c-framed segment log under _generations/facts/. The before-image history and the canonical tree remain authoritative; the fact log gives consumers ONE sequential, self-verifying stream (index heals, incremental replays) in place of a per-entity directory walk. - Wire format: positional msgpack facts [generation, timestamp, ops, meta, blobHashes]; op = [kind u8, id bin16, record | nil tombstone]; 32-byte segment header (magic, formatVersion, firstGeneration, zeroed+verified reserved); length+crc32c frame per fact; zero-padded segment names so lexicographic order == generation order; JSON manifest with an atomic rename flip, manifest-first rotation. - Commit protocol: facts append+fsync BEFORE the commit point inside the existing durability window, so a crash can only leave the log AHEAD of committed truth — open() truncates back (torn tails detected by CRC). Absent generation = never committed; a scan can never see an uncommitted fact. transact() facts are durable-on-return; single-op facts ride the group-commit flush exactly like buffered history. A fact-append failure fails the write, loudly — a silent gap would be a lie a later replay discovers. - New public surface: brain.scanFacts() (sequential batches with heal telemetry: head/segments/approx up front, per-batch generation range + bytes + segment id, loud abort on gaps, summary cross-check) and brain.factSegmentPaths() (immutable sealed segments for zero-copy consumers; the mutable tail excluded). Exported types CommitFact, FactOp, FactScanBatch, FactScanHandle. - Storage: optional binary raw-byte primitives (appendRawBytes, readRawBytes, writeRawBytes, rawByteSize) on StorageAdapter — feature-detected; filesystem + memory adapters implement them; an adapter without them hosts no fact log. Fact segments are byte-copied (never hard-linked) into snapshots. The _generations/facts/ namespace is registered as a protected family (rebuildable: false): no sweeper or GC may delete under it. - New crc32c (Castagnoli) utility with RFC known-answer tests. |
||
|---|---|---|
| .. | ||
| aggregation | ||
| brainy | ||
| db | ||
| graph | ||
| hnsw | ||
| import | ||
| indexes/columnStore | ||
| neural | ||
| performance | ||
| storage | ||
| transaction | ||
| types | ||
| universal | ||
| utils | ||
| vfs | ||
| boundary-no-native.test.ts | ||
| brainy-core.unit.test.ts | ||
| brainy-get-optimization.test.ts | ||
| cold-open-rebuild-gate.test.ts | ||
| create-entities-default.test.ts | ||
| fieldTypeInference.test.ts | ||
| get-index-status-readiness.test.ts | ||
| metadata-cold-read-guard.test.ts | ||
| migration-lock.test.ts | ||
| plugin-autodetect.test.ts | ||
| plugin-version-coupling.test.ts | ||
| plugin.test.ts | ||
| process-exit-sweep.test.ts | ||
| shutdown-hooks-lifecycle.test.ts | ||
| test-suite-coverage-guard.test.ts | ||
| type-filtering.unit.test.ts | ||
| validate-invariants-delegation.test.ts | ||
| vector-cold-read-guard.test.ts | ||
| vfs-multi-instance-diagnostic.test.ts | ||
| vfs-restart-fix.test.ts | ||
| where-operator-validation.test.ts | ||