fix(close): a read-only brain writes nothing under _system/
`readonly-close-no-marker` closed the clean-shutdown-marker half of this law and named the rest as a known residual. This is that residual, closed. MEASURED on the base: a read-only open → read → close rewrote FOUR files — `_system/__metadata_field_registry__.json.gz`, `type-statistics.json.gz`, `subtype-statistics.json.gz` and `verb-subtype-statistics.json.gz`. An IDLE reader that only opened and closed rewrote all four as well. The cause was not the closes the marker fix guarded. It was Phase 1 of closeDurableSteps, where every component flush ran unconditionally. A flush is a write by definition: MetadataIndexManager#flush() saves the field registry "even with no dirty fields" (its own comment), and the storage adapter's count flush re-stamps the three statistics files. A session that committed nothing re-stamped all four. Phase 2's closes were ungated too — the graph index's close drains both LSM MemTables to SSTables and stamps its watermark, and the optional vector/metadata `close` hooks (unimplemented in the reference engine, filled in by a native provider) persist buffered state. Every one of those calls now carries the same `!isReadOnly` guard the generation store already had. A reader still RELEASES what it holds, so Phase 2 is a branch rather than a skip: GraphAdjacencyIndex gains `stopBackgroundFlush()`, the non-writing half of its close, which clears the auto-flush interval that would otherwise outlive the session. `close()` now calls it too, so there is one place that owns the timer. Why this matters beyond tidiness: `_system/` is where a store keeps its evidence about itself — what the writer committed, what the projections have seen. A reader that rewrites any of it vouches for a state it only observed, and on shared or snapshot storage it mutates bytes another process owns. The pin hashes every file under `_system/` (and, in one case, the whole store) across a reader's open → read → close, names the four paths that used to move so a regression says which subsystem did it, and asserts the asymmetry holds in the other direction — a WRITER's close still persists.
This commit is contained in:
parent
0d5ab6077d
commit
f27a777615
4 changed files with 322 additions and 16 deletions
|
|
@ -20834,34 +20834,45 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
|
||||
// Phase 1: Flush ALL components in parallel to persist buffered data
|
||||
// This is critical when cor native providers buffer data in Rust memory
|
||||
//
|
||||
// READ-ONLY GUARD, applied to EVERY flush here. A flush is a write by
|
||||
// definition, and a reader has nothing of its own to persist — but these
|
||||
// calls were not conditional, so a read-only open → read → close REWROTE
|
||||
// four files under `_system/`: the metadata field registry (whose flush()
|
||||
// saves it unconditionally, "even with no dirty fields"), and the three
|
||||
// type/subtype statistics files the storage adapter's count flush stamps.
|
||||
// Every one of them was re-stamped on a session that committed nothing.
|
||||
// A reader must leave `_system/` exactly as it found it — the same law the
|
||||
// clean-shutdown marker already lives under (see the generation-store
|
||||
// guard below and `Brainy.openReadOnly`).
|
||||
await Promise.all([
|
||||
// Flush HNSW dirty nodes (deferred persistence mode)
|
||||
(async () => {
|
||||
if (this.index && typeof this.index.flush === 'function') {
|
||||
if (this.index && !this.isReadOnly && typeof this.index.flush === 'function') {
|
||||
await this.index.flush()
|
||||
}
|
||||
})(),
|
||||
// Flush metadata index (field indexes + EntityIdMapper)
|
||||
(async () => {
|
||||
if (this.metadataIndex && typeof this.metadataIndex.flush === 'function') {
|
||||
if (this.metadataIndex && !this.isReadOnly && typeof this.metadataIndex.flush === 'function') {
|
||||
await this.metadataIndex.flush()
|
||||
}
|
||||
})(),
|
||||
// Flush graph adjacency index (LSM trees)
|
||||
(async () => {
|
||||
if (this.graphIndex && typeof this.graphIndex.flush === 'function') {
|
||||
if (this.graphIndex && !this.isReadOnly && typeof this.graphIndex.flush === 'function') {
|
||||
await this.graphIndex.flush()
|
||||
}
|
||||
})(),
|
||||
// Flush storage adapter counts
|
||||
(async () => {
|
||||
if (this.storage && typeof this.storage.flushCounts === 'function') {
|
||||
if (this.storage && !this.isReadOnly && typeof this.storage.flushCounts === 'function') {
|
||||
await this.storage.flushCounts()
|
||||
}
|
||||
})(),
|
||||
// Flush aggregation index state
|
||||
(async () => {
|
||||
if (this._aggregationIndex) {
|
||||
if (this._aggregationIndex && !this.isReadOnly) {
|
||||
await this._aggregationIndex.flush()
|
||||
}
|
||||
})(),
|
||||
|
|
@ -20910,21 +20921,37 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
|
||||
// Phase 2: Close components to release resources (timers, file handles)
|
||||
// Data is already safe on disk from Phase 1
|
||||
//
|
||||
// READ-ONLY GUARD, same law as Phase 1. Each of these closes is a WRITER:
|
||||
// the graph index drains both LSM MemTables to SSTables and stamps its
|
||||
// watermark, and the vector/metadata `close` hooks — optional doors the
|
||||
// reference engine leaves unimplemented, but which a native provider fills
|
||||
// in — persist their buffered state. None of that is a reader's to write.
|
||||
//
|
||||
// A reader still has to RELEASE what it holds, which is why this is a
|
||||
// branch rather than a skip: `stopBackgroundFlush()` is the non-writing
|
||||
// half of the graph index's close, clearing the auto-flush interval that
|
||||
// would otherwise outlive the session. The optional hooks have no
|
||||
// non-writing counterpart to call, and a provider that buffers nothing on
|
||||
// a read-only open has nothing to release.
|
||||
await Promise.all([
|
||||
(async () => {
|
||||
if (this.graphIndex && typeof this.graphIndex.close === 'function') {
|
||||
if (!this.graphIndex) return
|
||||
if (this.isReadOnly) {
|
||||
this.graphIndex.stopBackgroundFlush()
|
||||
} else if (typeof this.graphIndex.close === 'function') {
|
||||
await this.graphIndex.close()
|
||||
}
|
||||
})(),
|
||||
(async () => {
|
||||
const index = this.index as JsHnswVectorIndex & VectorIndexOptionalHooks
|
||||
if (index && typeof index.close === 'function') {
|
||||
if (index && !this.isReadOnly && typeof index.close === 'function') {
|
||||
await index.close()
|
||||
}
|
||||
})(),
|
||||
(async () => {
|
||||
const metadataIndex = this.metadataIndex as MetadataIndexManager & MetadataIndexOptionalHooks
|
||||
if (metadataIndex && typeof metadataIndex.close === 'function') {
|
||||
if (metadataIndex && !this.isReadOnly && typeof metadataIndex.close === 'function') {
|
||||
await metadataIndex.close()
|
||||
}
|
||||
})(),
|
||||
|
|
|
|||
|
|
@ -1105,13 +1105,31 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
}
|
||||
|
||||
/**
|
||||
* Clean shutdown
|
||||
* Stop the auto-flush interval WITHOUT writing anything.
|
||||
*
|
||||
* The non-writing half of {@link close}, for a shutdown that must leave the
|
||||
* store byte-identical — a read-only brain's close. `close()` itself is a
|
||||
* writer: it drains both LSM MemTables to SSTables and stamps the watermark,
|
||||
* which is exactly right for a writer and forbidden for a reader. A reader
|
||||
* still has to release this interval, though: it is the one piece of this
|
||||
* index that outlives the close and could fire against a store the session no
|
||||
* longer owns.
|
||||
*
|
||||
* @returns Nothing.
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
stopBackgroundFlush(): void {
|
||||
if (this.flushTimer) {
|
||||
clearInterval(this.flushTimer)
|
||||
this.flushTimer = undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean shutdown — drains both trees and stamps the watermark. THIS WRITES;
|
||||
* a read-only brain must call {@link stopBackgroundFlush} instead.
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
this.stopBackgroundFlush()
|
||||
|
||||
// Close both LSM-trees (will flush MemTables to SSTables)
|
||||
if (this.initialized) {
|
||||
|
|
|
|||
Reference in a new issue