fix: count recovery scans the canonical layout; remove the dead 7.x hnsw sharding machinery
Sweeping the vestigial hnsw sharding machinery surfaced two real defects on the counts-recovery path (the code that rebuilds totalNounCount/totalVerbCount when counts.json is lost or corrupted — container restarts, partial copies): - initializeCountsFromDisk counted files in entities/*/hnsw/ — directories the 8.0 write path never populates — so an established store recovered to ZERO counts on real data: wrong getNounCount()/stats, a "New installation"-style boot log, and a mis-sized rebuild-strategy decision at open. The scan now walks the canonical entities/<kind>/<shard>/<id>/ tree (one entity per id directory), with the same sampled type-distribution estimate as before. - The sampler called getNounMetadata — whose ensureInitialized() re-enters init() — from INSIDE init(), a latent deadlock reachable the moment the scan found anything to sample. The sampled metadata files are now read directly with fs (gz-transparent), no guarded accessors inside init. The dead machinery itself (verified zero callers by reachability analysis before deletion): the nine 7.x hnsw-layout entity/edge CRUD methods, the sharding-depth prober + its depth-migration engine (unreachable — the probe always returned null on 8.0 stores), an orphaned streaming verb paginator, their path/scan helpers, and the now-unused type aliases and fields. The init block keeps the truthful new-vs-established boot log introduced in 8.0.13, now decided directly from the canonical layout. Net -1,138 lines; no public API touched. Adds a regression test: delete counts.json from a populated store, reopen, counts recover exactly from the canonical tree.
This commit is contained in:
parent
716a8513bf
commit
352e2da88f
2 changed files with 127 additions and 1150 deletions
File diff suppressed because it is too large
Load diff
|
|
@ -87,6 +87,29 @@ describe('FileSystemStorage boot log: an established brain is not "New installat
|
|||
await teardown(second)
|
||||
})
|
||||
|
||||
it('counts RECOVER from the canonical layout when counts.json is lost (container-restart case)', async () => {
|
||||
const dir = mkTmp()
|
||||
|
||||
// Establish a store with 3 entities (canonical layout + counts.json).
|
||||
const first: any = new FileSystemStorage(dir)
|
||||
await first.init()
|
||||
for (const c of ['a', 'b', 'c']) await seedOne(first, c.repeat(32))
|
||||
await teardown(first)
|
||||
|
||||
// Simulate the lost/corrupted counts.json a container restart can leave.
|
||||
for (const f of ['counts.json', 'counts.json.gz']) {
|
||||
rmSync(join(dir, '_system', f), { force: true })
|
||||
}
|
||||
|
||||
// Reopen: the recovery scan must count the CANONICAL tree. The removed
|
||||
// implementation scanned the vestigial `entities/*/hnsw` dirs and
|
||||
// re-initialized every counter to ZERO on real data.
|
||||
const second: any = new FileSystemStorage(dir)
|
||||
await second.init()
|
||||
expect(await second.getNounCount()).toBe(3)
|
||||
await teardown(second)
|
||||
})
|
||||
|
||||
it('hasCanonicalEntities() sees the canonical shard tree — independent of counts.json', async () => {
|
||||
const dir = mkTmp()
|
||||
const s: any = new FileSystemStorage(dir)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue