fix(8.0): per-type counts rehydrate after cold reopen (column store, not dead sparse index)

lazyLoadCounts() read the `__sparse_index__noun` blob, but the sparse-index
WRITE path was removed in 7.20.0 — new workspaces persist the 'noun' field
ONLY to the column store. So on close()+reopen the sparse load found nothing
and left every per-type count at 0: counts.byType / byTypeEnum / topTypes /
allNounTypeCounts all read empty, while find() / getNounCount() (different
sources) stayed correct.

Rehydrate from the column store's 'noun' field instead. Its per-value
cardinality matches the warm updateTypeFieldAffinity counts exactly because
both are driven from the same addToIndex field set, in lockstep, with no
visibility gate on either — so syncTypeCountsToFixed (called right after in
init) reproduces the warm fixed-array values precisely. Legacy chunked sparse
index kept as a fallback for pre-7.20.0 workspaces.

Ground-truth verified: 12 Person + 5 Document → cold reopen now reports
person:12 / document:5 (was 0), topTypes [person, document, collection].

Tests: un-skipped the intentionally-failing phase1c "warm cache on init"
reopen test and strengthened it to exact persisted counts; added a warm==cold
element-for-element equality test. 1464 unit + count-sync/multi-process/
clear-persistence integration green.
This commit is contained in:
David Snelling 2026-06-19 10:55:43 -07:00
parent 1264fec534
commit d918f49287
2 changed files with 92 additions and 14 deletions

View file

@ -377,23 +377,69 @@ describe('Brainy - Phase 1c: Type-Aware Integration', () => {
silent: true
})
await brainy2.init() // init() is expected to rehydrate type counts from the persisted noun index
await brainy2.init() // init() rehydrates type counts from the persisted column store
try {
// After reopening a persisted brain, counts.topTypes() must reflect the
// stored data. NOTE (8.0): this currently FAILS — the metadataIndex-backed
// counts.* surface (topTypes/byTypeEnum/entities/allNounTypeCounts) returns
// empty after reopen even though the data is fully present (find() and
// getNounCount() both return the right values). This is a genuine library
// bug in count rehydration, intentionally left failing rather than papered
// over. See the agent's realBugs report for the precise repro.
// stored data. Regression guard for the 8.0 cold-reopen count bug:
// lazyLoadCounts read the dead `__sparse_index__noun` blob (sparse WRITE
// path removed in 7.20.0) and left every per-type count at 0, so
// counts.topTypes/byTypeEnum/allNounTypeCounts returned empty after reopen
// even though find()/getNounCount() were correct. Fixed by rehydrating
// from the column store's 'noun' field.
const topTypes = brainy2.counts.topTypes(3)
expect(topTypes[0]).toBe('person') // Most common type
expect(topTypes[1]).toBe('document')
// Counts must rehydrate to the EXACT persisted values, not just be ordered.
expect(brainy2.counts.byTypeEnum('person')).toBe(100)
expect(brainy2.counts.byTypeEnum('document')).toBe(50)
expect(await brainy2.counts.byType('person')).toBe(100)
const allNoun = brainy2.counts.allNounTypeCounts()
expect(allNoun.get('person' as any)).toBe(100)
expect(allNoun.get('document' as any)).toBe(50)
} finally {
await brainy2.close()
}
})
it('rehydrated per-type counts after cold reopen equal the warm counts exactly', async () => {
// Audit mandate: add N of a type → byTypeEnum(t) === N, both WARM and after
// a close()+reopen, with warm and cold reporting identical maps.
for (let i = 0; i < 7; i++) {
await brainy.add({ data: `Person ${i}`, type: NounType.Person, metadata: { name: `P${i}` } })
}
for (let i = 0; i < 3; i++) {
await brainy.add({ data: `Task ${i}`, type: NounType.Task, metadata: { title: `T${i}` } })
}
await brainy.flush()
// Capture the warm (in-session) counts before closing.
const warmPerson = brainy.counts.byTypeEnum('person')
const warmTask = brainy.counts.byTypeEnum('task')
const warmAll = Object.fromEntries(brainy.counts.allNounTypeCounts() as Map<string, number>)
expect(warmPerson).toBe(7)
expect(warmTask).toBe(3)
const reopened = new Brainy({
requireSubtype: false,
storage: { type: 'filesystem', rootDirectory: testDir },
dimensions: 384,
silent: true
})
await reopened.init()
try {
// Cold counts equal the exact persisted values...
expect(reopened.counts.byTypeEnum('person')).toBe(7)
expect(reopened.counts.byTypeEnum('task')).toBe(3)
// ...and equal the warm counts map element-for-element.
const coldAll = Object.fromEntries(reopened.counts.allNounTypeCounts() as Map<string, number>)
expect(coldAll).toEqual(warmAll)
} finally {
await reopened.close()
}
})
})
describe('Performance Characteristics', () => {