feat(open-path): init never gates on the embedding model; open goes concurrent; slow opens narrate
A production restart storm measured 90,017ms for a single brain init vs 1,117ms quiet (~80x contention multiplier), traced to performInit() eagerly awaiting the process-global WASM embedding engine before the VFS root even existed. Every writer's open() queued on the one throttled model compile (90-140s on throttled CPUs). - VirtualFileSystem.doInitializeRoot() no longer embeds '/'. The root is system-tier plumbing nothing ever searches; when the default WASM engine is active it now gets an explicit all-zero placeholder vector (cosineDistance returns max distance for a zero vector, so it never ranks ahead of real content). deferEmbedding was considered and rejected: its landing path kicks the embed worker synchronously right after commit, which would still force the cold compile within milliseconds — just off the awaited path, not avoided. A registered native 'embeddings' provider (no cold-start cost, possibly a different dimension) still embeds the root for real, via the new Brainy.usesDefaultWasmEmbedder() seam. - performInit()'s eager-embedding step now only STARTS the WASM engine warm in the background instead of awaiting it inline. embed()/embeddingManager already serialize concurrent callers on one shared init promise, so the first real embed() converges correctly either way; a failed warm narrates loudly instead of surfacing as a silent latency spike or an unhandled rejection. eagerEmbeddings: false still means no warm at all. - FileSystemStorage.init() batches its ~8 independent bootstrap mkdirs (each creates its own full subtree via recursive:true, so none depend on the others existing) into one Promise.all. The restore-completion step and initializeCounts() stay strictly sequential — they have real order dependencies on rootDir and systemDir respectively. - performInit() now times five phases (storage init / generation-store open+fold / index init+gate / VFS bootstrap / embedding-warm-started) and logs one warning with the per-phase breakdown when total open exceeds 2000ms; silent otherwise.
This commit is contained in:
parent
f8f64780b1
commit
96624f408c
5 changed files with 446 additions and 53 deletions
|
|
@ -239,38 +239,54 @@ export class FileSystemStorage extends BaseStorage {
|
|||
|
||||
// Finish any restore interrupted by a crash (resume the staged swap, or
|
||||
// discard an uncommitted staging area) BEFORE counts/derived state load,
|
||||
// so the rest of startup sees the completed store.
|
||||
// so the rest of startup sees the completed store. ORDER-DEPENDENT:
|
||||
// `swapStagedRestoreIn()` reads `fs.readdir(rootDir)` and then
|
||||
// removes/renames rootDir's own TOP-LEVEL entries to place the staged
|
||||
// copy — racing that against the directory-creation batch below (which
|
||||
// also touches rootDir's children) could see a half-created directory
|
||||
// mid-swap or a mkdir racing a concurrent rm/rename on the same path.
|
||||
// Stays strictly sequential, never folded into the OPEN-PATH batch.
|
||||
await this.completeInterruptedRestore()
|
||||
|
||||
// Create the nouns directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.nounsDir)
|
||||
// OPEN-PATH FIX: the remaining bootstrap directories are mutually
|
||||
// independent — each is its own subtree under rootDir, and
|
||||
// `fs.mkdir(dir, { recursive: true })` creates every intermediate
|
||||
// segment of ITS OWN path in one call, so it never depends on any
|
||||
// sibling here existing first. Nothing between here and
|
||||
// `initializeCounts()` reads any of them, so batching collapses what
|
||||
// was up to 8 sequential mkdir round-trips (each a real syscall+await)
|
||||
// into one wave — this is what serialized an N-writer restart storm on
|
||||
// filesystem I/O it never structurally needed. `initializeCounts()`
|
||||
// right after DOES depend on `systemDir` (which the batch creates), so
|
||||
// it stays outside, awaited only once every directory has landed.
|
||||
await Promise.all([
|
||||
// Create the nouns directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.nounsDir),
|
||||
// Create the verbs directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.verbsDir),
|
||||
// Create the metadata directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.metadataDir),
|
||||
// Create the noun metadata directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.nounMetadataDir),
|
||||
// Create the verb metadata directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.verbMetadataDir),
|
||||
// Create both directories for backward compatibility
|
||||
this.ensureDirectoryExists(this.systemDir),
|
||||
// Only create legacy directory if it exists (don't create new legacy
|
||||
// dirs) — a read-then-maybe-write, but on its own subtree, so it's
|
||||
// still independent of every other entry in this batch.
|
||||
(async () => {
|
||||
if (await this.directoryExists(this.indexDir)) {
|
||||
await this.ensureDirectoryExists(this.indexDir)
|
||||
}
|
||||
})(),
|
||||
// Create the locks directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.lockDir),
|
||||
// Create the binary blobs directory if it doesn't exist
|
||||
this.ensureDirectoryExists(this.blobsDir)
|
||||
])
|
||||
|
||||
// Create the verbs directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.verbsDir)
|
||||
|
||||
// Create the metadata directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.metadataDir)
|
||||
|
||||
// Create the noun metadata directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.nounMetadataDir)
|
||||
|
||||
// Create the verb metadata directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.verbMetadataDir)
|
||||
|
||||
// Create both directories for backward compatibility
|
||||
await this.ensureDirectoryExists(this.systemDir)
|
||||
// Only create legacy directory if it exists (don't create new legacy dirs)
|
||||
if (await this.directoryExists(this.indexDir)) {
|
||||
await this.ensureDirectoryExists(this.indexDir)
|
||||
}
|
||||
|
||||
// Create the locks directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.lockDir)
|
||||
|
||||
// Create the binary blobs directory if it doesn't exist
|
||||
await this.ensureDirectoryExists(this.blobsDir)
|
||||
|
||||
// Initialize count management
|
||||
// Initialize count management — depends on systemDir, created above.
|
||||
this.countsFilePath = path.join(this.systemDir, 'counts.json')
|
||||
await this.initializeCounts()
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue