fix: an established store no longer boot-logs "New installation"
Every persisted 8.0 store logged "📁 New installation: using depth 1 sharding"
on every open — even brains holding thousands of entities — which is alarming to
read during a restart or incident. 8.0 stores nouns in the canonical
`entities/nouns/<shard>/<id>/vectors.json` layout (the path `saveNoun`/`getNouns`
use), but the legacy sharding probe `detectExistingShardingDepth()` inspects
`entities/nouns/hnsw` — a 7.x directory the 8.0 write path never populates (its
only writer, `saveNode`, is dead code with zero callers). So the probe returned
null for every 8.0 store and concluded "new".
Drive the new-vs-existing log from the layout the database actually reads and
writes: a new `hasCanonicalEntities()` checks for a real 2-hex shard directory
under `entities/nouns/`, and the known noun count short-circuits it. An
established store now logs "Using depth 1 sharding (N entities)"; only a genuinely
empty store reports a new installation. Behavior is otherwise unchanged — the
probe only ever set a log line, never triggered a rebuild or migration.
This commit is contained in:
parent
4341272c56
commit
308691603f
2 changed files with 151 additions and 2 deletions
|
|
@ -265,8 +265,19 @@ export class FileSystemStorage extends BaseStorage {
|
|||
|
||||
console.log(`✅ Migration complete - now using depth ${this.SHARDING_DEPTH} sharding`)
|
||||
} else if (detectedDepth === null) {
|
||||
// New installation
|
||||
console.log(`📁 New installation: using depth ${this.SHARDING_DEPTH} sharding (optimal for 1-2.5M entities)`)
|
||||
// The legacy sharding probe inspects `entities/nouns/hnsw` — a 7.x
|
||||
// directory the 8.0 write path never populates (8.0 stores entities in
|
||||
// the canonical `entities/nouns/<shard>/<id>/` layout), so it returns
|
||||
// null for EVERY 8.0 store, new or not. Decide new-vs-existing from the
|
||||
// canonical layout the DB actually reads/writes so an established brain
|
||||
// is not mislabeled "New installation" on every boot.
|
||||
const established =
|
||||
this.totalNounCount > 0 || (await this.hasCanonicalEntities())
|
||||
console.log(
|
||||
established
|
||||
? `📁 Using depth ${this.SHARDING_DEPTH} sharding (${this.totalNounCount} entities)`
|
||||
: `📁 New installation: using depth ${this.SHARDING_DEPTH} sharding (optimal for 1-2.5M entities)`
|
||||
)
|
||||
} else {
|
||||
// Already using correct depth
|
||||
console.log(`📁 Using depth ${this.SHARDING_DEPTH} sharding (${this.totalNounCount} entities)`)
|
||||
|
|
@ -2737,6 +2748,39 @@ export class FileSystemStorage extends BaseStorage {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this store already holds canonical 8.0 entities.
|
||||
*
|
||||
* 8.0 writes nouns to `entities/nouns/<shard>/<id>/vectors.json` (see
|
||||
* `getNounVectorPath`), but the legacy sharding probe
|
||||
* ({@link detectExistingShardingDepth}) inspects `entities/nouns/hnsw` — a 7.x
|
||||
* directory the 8.0 write path never populates. The probe therefore returns
|
||||
* null for every 8.0 store and cannot tell an established brain from a fresh
|
||||
* one, which mislabels established stores "New installation" on every boot.
|
||||
* This checks the canonical shard tree the DB actually reads and writes (the
|
||||
* same `entities/nouns/<hex>` shards `getNounsWithPagination` walks) so boot
|
||||
* logs are truthful.
|
||||
*
|
||||
* @returns true if at least one 2-hex shard directory (00–ff) exists under
|
||||
* `entities/nouns/`, i.e. the store has previously persisted entities.
|
||||
*/
|
||||
private async hasCanonicalEntities(): Promise<boolean> {
|
||||
const canonicalNounsDir = path.join(this.rootDir, 'entities', 'nouns')
|
||||
try {
|
||||
const entries = await fs.promises.readdir(canonicalNounsDir, {
|
||||
withFileTypes: true
|
||||
})
|
||||
// A populated store has ≥1 two-hex shard dir (00–ff). The vestigial
|
||||
// `hnsw` subdir is 4 chars and correctly excluded by the hex test.
|
||||
return entries.some(
|
||||
(e: any) => e.isDirectory() && /^[0-9a-f]{2}$/i.test(e.name)
|
||||
)
|
||||
} catch {
|
||||
// Directory absent (fresh store) or unreadable → no persisted entities.
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get sharding depth
|
||||
* Always returns 1 (single-level sharding) for optimal balance of
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue