soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

fix(8.0): stats() per-type counts no longer inflate with HNSW re-saves

Browse source 
nounCountsByType (read by stats().entitiesByType / counts.byTypeEnum) was
incremented in saveNoun_internal(), which runs on every noun write — including
the HNSW index re-saving a node whenever its neighbor links change. So per-type
counts grew with graph connectivity instead of entity count (an internal report
saw 8 documents read as 44). Moved the increment into saveNounMetadata_internal(),
gated on isNew + visibility, exactly parallel to the authoritative total; the
visibility-flip path adjusts it in lockstep too.

Tests: tests/unit/storage/stats-count-accuracy.test.ts (30 dense-type entities ->
exactly 30 across stats/counts/getNounCount) + de-theatricalized the >=2 assertion
in brainy-core.unit.test.ts to exact equality. Build green; 1455 unit green.

Part of the 8.0 readiness audit Phase 1; cold-reopen count rehydration is still WIP.
This commit is contained in:
David Snelling 2026-06-17 17:07:58 -07:00
parent b2005ff22a
commit 21d02d3bae
3 changed files with 79 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

27
src/storage/baseStorage.ts
View file

@ -2462,18 +2462,27 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// 8.0: skip the user-facing total for internal/system entities (counts.json + getNounCount()).
if (isNew && metadata.noun && isCounted) {
this.incrementEntityCount(metadata.noun)
// Per-type counter (stats().entitiesByType / counts.byTypeEnum) is maintained
// HERE — gated on isNew + visibility, exactly parallel to the total above. It
// used to be bumped unconditionally in saveNoun_internal(), but the HNSW index
// re-saves a node on every neighbor-link change, so that inflated the per-type
// counts with graph connectivity (e.g. 8 documents could read as 44).
this.nounCountsByType[TypeUtils.getNounIndex(metadata.noun as NounType)]++
// Persist counts asynchronously (fire and forget)
this.scheduleCountPersist().catch(() => {
// Ignore persist errors - will retry on next operation
})
} else if (!isNew && metadata.noun && wasCounted !== isCounted) {
// Visibility flipped on update(): move the entity in/out of the user-facing total
// (counts.json / getNounCount()). `nounCountsByType` is gated independently in
// `saveNoun_internal()` off the cache warmed just above, so it is not touched here.
// Visibility flipped on update(): move the entity in/out of the user-facing
// total (counts.json / getNounCount()) AND the per-type counter together, so
// stats().entitiesByType stays consistent with getNounCount().
const typeIdx = TypeUtils.getNounIndex(metadata.noun as NounType)
if (isCounted) {
this.incrementEntityCount(metadata.noun)
this.nounCountsByType[typeIdx]++
} else {
this.decrementEntityCount(metadata.noun)
if (this.nounCountsByType[typeIdx] > 0) this.nounCountsByType[typeIdx]--
}
this.scheduleCountPersist().catch(() => {})
}
@ -3656,15 +3665,13 @@ export abstract class BaseStorage extends BaseStorageAdapter {
const type = this.getNounType(noun)
const path = getNounVectorPath(noun.id)
// Update type tracking — but only for entities that count toward the
// user-facing per-type stats. `nounVisibilityByIdCache` (warmed by
// `saveNounMetadata_internal()`) flags internal/system ids; public ids are
// absent, so the common case still increments. 8.0 visibility exclusion.
// Per-type stats counter (`nounCountsByType`, read by stats().entitiesByType)
// is maintained in saveNounMetadata_internal(), gated on isNew + visibility —
// NOT here. saveNoun_internal() also runs on HNSW neighbor-link re-saves, so
// incrementing here inflated the per-type counts with graph connectivity. We
// only READ the (externally-maintained) count below to decide when to persist.
const typeIndex = TypeUtils.getNounIndex(type)
const counted = isCountedVisibility(this.nounVisibilityByIdCache.get(noun.id))
if (counted) {
this.nounCountsByType[typeIndex]++
}
// Write-cache coherent canonical write
await this.writeCanonicalObject(path, noun)