fix: count recovery scans the canonical layout; remove the dead 7.x hnsw sharding machinery

Browse source

Sweeping the vestigial hnsw sharding machinery surfaced two real defects on
the counts-recovery path (the code that rebuilds totalNounCount/totalVerbCount
when counts.json is lost or corrupted — container restarts, partial copies):

- initializeCountsFromDisk counted files in entities/*/hnsw/ — directories the
  8.0 write path never populates — so an established store recovered to ZERO
  counts on real data: wrong getNounCount()/stats, a "New installation"-style
  boot log, and a mis-sized rebuild-strategy decision at open. The scan now
  walks the canonical entities/<kind>/<shard>/<id>/ tree (one entity per id
  directory), with the same sampled type-distribution estimate as before.
- The sampler called getNounMetadata — whose ensureInitialized() re-enters
  init() — from INSIDE init(), a latent deadlock reachable the moment the scan
  found anything to sample. The sampled metadata files are now read directly
  with fs (gz-transparent), no guarded accessors inside init.

The dead machinery itself (verified zero callers by reachability analysis
before deletion): the nine 7.x hnsw-layout entity/edge CRUD methods, the
sharding-depth prober + its depth-migration engine (unreachable — the probe
always returned null on 8.0 stores), an orphaned streaming verb paginator,
their path/scan helpers, and the now-unused type aliases and fields. The init
block keeps the truthful new-vs-established boot log introduced in 8.0.13,
now decided directly from the canonical layout. Net -1,138 lines; no public
API touched.

Adds a regression test: delete counts.json from a populated store, reopen,
counts recover exactly from the canonical tree.

...

This commit is contained in:

David Snelling 2026-07-08 15:49:19 -07:00

parent 716a8513bf

commit 352e2da88f

2 changed files with 127 additions and 1150 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1254

src/storage/adapters/fileSystemStorage.ts

View file

File diff suppressed because it is too large Load diff