fix(8.0): per-type counts rehydrate after cold reopen (column store, not dead sparse index)
lazyLoadCounts() read the `__sparse_index__noun` blob, but the sparse-index WRITE path was removed in 7.20.0 — new workspaces persist the 'noun' field ONLY to the column store. So on close()+reopen the sparse load found nothing and left every per-type count at 0: counts.byType / byTypeEnum / topTypes / allNounTypeCounts all read empty, while find() / getNounCount() (different sources) stayed correct. Rehydrate from the column store's 'noun' field instead. Its per-value cardinality matches the warm updateTypeFieldAffinity counts exactly because both are driven from the same addToIndex field set, in lockstep, with no visibility gate on either — so syncTypeCountsToFixed (called right after in init) reproduces the warm fixed-array values precisely. Legacy chunked sparse index kept as a fallback for pre-7.20.0 workspaces. Ground-truth verified: 12 Person + 5 Document → cold reopen now reports person:12 / document:5 (was 0), topTypes [person, document, collection]. Tests: un-skipped the intentionally-failing phase1c "warm cache on init" reopen test and strengthened it to exact persisted counts; added a warm==cold element-for-element equality test. 1464 unit + count-sync/multi-process/ clear-persistence integration green.
This commit is contained in:
parent
1264fec534
commit
d918f49287
2 changed files with 92 additions and 14 deletions
|
|
@ -508,10 +508,41 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
this.entityCountsByTypeFixed.fill(0)
|
||||
this.verbCountsByTypeFixed.fill(0)
|
||||
|
||||
// Load counts from sparse index (correct source)
|
||||
// PRIMARY (8.0+): rehydrate per-type counts from the column store's 'noun'
|
||||
// field — the authoritative on-disk source after a cold reopen.
|
||||
//
|
||||
// The chunked sparse-index WRITE path was removed in 7.20.0 (commit
|
||||
// 11be039): new workspaces persist the 'noun' field ONLY to the column
|
||||
// store, never to a `__sparse_index__noun` blob. So the legacy sparse
|
||||
// path below finds nothing and leaves every count at 0 — which is exactly
|
||||
// why counts.byType/byTypeEnum/topTypes/allNounTypeCounts all read empty
|
||||
// after close()+reopen while find()/getNounCount() (different sources)
|
||||
// stay correct. The column store's per-value cardinality matches the warm
|
||||
// `updateTypeFieldAffinity` counts EXACTLY because both are driven from the
|
||||
// same `addToIndex` field set, in lockstep, with no visibility gate on
|
||||
// either — so this rehydration reproduces the warm values precisely.
|
||||
if (this.columnStore && this.columnStore.getIndexedFields().includes('noun')) {
|
||||
const nounValues = await this.columnStore.getFilterValues('noun')
|
||||
for (const value of nounValues) {
|
||||
const bitmap = await this.columnStore.filter('noun', value)
|
||||
if (bitmap.size > 0) {
|
||||
// Use the stored value directly as the key (the legacy sparse path
|
||||
// did the same): it is already the normalized type string that
|
||||
// getNounFromIndex/getEntityCountByType expect, so syncTypeCountsToFixed
|
||||
// — called immediately after lazyLoadCounts in init() — copies it into
|
||||
// entityCountsByTypeFixed without re-normalization drift.
|
||||
this.totalEntitiesByType.set(value, bitmap.size)
|
||||
}
|
||||
}
|
||||
prodLog.debug(`✅ Rehydrated type counts from column store: ${this.totalEntitiesByType.size} types`)
|
||||
return
|
||||
}
|
||||
|
||||
// LEGACY FALLBACK (pre-7.20.0 workspaces still on the chunked sparse index).
|
||||
const nounSparseIndex = await this.loadSparseIndex('noun')
|
||||
if (!nounSparseIndex) {
|
||||
// No sparse index yet - counts will be populated as entities are added
|
||||
// No column-store 'noun' field and no sparse index yet — counts will be
|
||||
// populated as entities are added.
|
||||
return
|
||||
}
|
||||
|
||||
|
|
@ -530,7 +561,7 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
} catch (error) {
|
||||
// Silently fail - counts will be populated as entities are added
|
||||
// This maintains zero-configuration principle
|
||||
prodLog.debug('Could not load type counts from sparse index:', error)
|
||||
prodLog.debug('Could not load type counts:', error)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -2684,10 +2715,11 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
}
|
||||
|
||||
/**
|
||||
* Get all entity types and their counts - O(1) operation
|
||||
* Fixed - totalEntitiesByType is correctly populated by updateTypeFieldAffinity
|
||||
* during add operations. lazyLoadCounts was reading wrong data but that doesn't
|
||||
* affect freshly-added entities within the same session.
|
||||
* Get all entity types and their counts - O(1) operation.
|
||||
* `totalEntitiesByType` is populated by `updateTypeFieldAffinity` during add
|
||||
* operations (warm path) and rehydrated from the column store's 'noun' field
|
||||
* by `lazyLoadCounts` on init (cold reopen), so this is accurate both within a
|
||||
* session and after close()+reopen.
|
||||
*/
|
||||
getAllEntityCounts(): Map<string, number> {
|
||||
return new Map(this.totalEntitiesByType)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue