Compare commits

..

2 commits

Author SHA1 Message Date
23a61cae70 test(shutdown): pin one owner per brain — real processes, real signals
Some checks are pending
CI / Node 22 (push) Waiting to run
CI / Node 24 (push) Waiting to run
CI / Integration + conformance (Node 22) (push) Waiting to run
CI / Bun (latest) (push) Waiting to run
Four pins in real child processes under real SIGTERM, following the
writer-lock-clean-close spawn pattern:

(a) A host owner registered on SIGTERM closes two brains while the engine's
    hooks are live: exactly one close entered and one close body run per brain,
    the writer lock given up exactly ONCE per brain, the handler announcing
    that it stepped aside, no "Writer fence lost", no failed instance, both
    durability markers written, exit 0, and both reopens adopting rather than
    folding. The release count is the discriminating assertion — against the
    old handler it reads {a: 2, b: 2}, one release from the owner's close and
    one from the handler's own finally.
(b) No host owner: the engine's handler closes every instance by the same
    path — one close each, markers written, clean exit, clean reopen.
(c) Two concurrent close() callers share one promise (by identity) and one
    execution; a third call after they settle runs nothing.
(d) Eight kicks during a running flush — five through the cadence door, three
    direct — arm exactly ONE follow-up: two flush bodies total, and the
    concurrency high-water mark stays at 1.

The counts come out of the child through a file written synchronously on the
way out: the engine calls process.exit(0) when it is the sole shutdown owner,
and a console.log to a pipe can be dropped by that exit.
2026-09-02 10:42:04 -07:00
f6f116fbb7 fix(shutdown): one owner per brain — the signal handler defers to close(), and flush is single-flight
MEASURED IN PRODUCTION. A host that owns its own shutdown — one SIGTERM
listener calling close() on every pooled store — ran head-on into the engine's
own signal handler, which iterated every live instance, flushed its components
in parallel, and released its writer lock in its own finally. Two teardowns of
the same brain at the same moment: "Shutdown signal received - flushing pending
data...", 148s of silence, "Flushed successfully (1 instance)", and the host's
pool close of that same store returning 1s later — 149s against 24s for the six
stores with no engine work in flight. The same race reproduced locally as
"Failed to flush one Brainy instance on shutdown: Writer fence lost … the lock
file is gone": the handler observing a lock the close it was racing had already
released.

Three changes, one law — a brain's teardown belongs to whoever started it.

1. close() is idempotent and re-entrant. The first call stores its promise
   synchronously in _closeInFlight and every later or concurrent caller gets
   that same promise back; the teardown runs once. close() is no longer async
   so the promise is shared by identity, not just outcome. The state is
   observable: isClosing (begun) and isClosed (finished).

2. The signal handler defers one macrotask, then per instance either steps
   aside (a close has begun or finished — its owner owns the flush, the markers
   and the lock) or awaits instance.close(): the same settle/flush/attest/
   marker/lock path any caller gets. Its old parallel per-component flush and
   separate lock release are gone; the three laws that block carried are each
   satisfied by close(), verified line by line and recorded in the new comment.
   Per-instance isolation stays here, in the loop's try/catch.

   Sole-owner exit now reads the listener count WHEN THE SIGNAL ARRIVES.
   Asking afterwards reads a process that has already torn itself down —
   closing the last brain deregisters the engine's own listeners, so a host's
   single remaining listener would look like "<= 1" and be force-exited out of
   its own graceful shutdown.

3. Flush is single-flight with a queue one deep. It did not coalesce: the
   cadence's guard covered only the flushes the cadence started, so a
   cross-process flush request or an application flush() overlapped it freely —
   production showed two "Flushing Brainy indexes…" runs 3s apart, walls
   growing 295ms to 4.9s. The gate now lives in flush() itself and covers every
   caller: run, or join the ONE queued follow-up. A follow-up rather than
   joining the running flush, because a caller flushes to make ITS writes
   durable and those may have landed after the running flush read its state; it
   costs nothing when there is nothing new. close() drains that chain too.

The idle law is untouched: a clean brain's flush still returns immediately, and
an idle brain still flushes zero times.
2026-09-02 10:41:55 -07:00
2 changed files with 5 additions and 233 deletions

View file

@ -1572,19 +1572,7 @@ export class VirtualFileSystem implements IVirtualFileSystem {
// ============= Semantic Operations =============
/**
* Search files with natural language.
*
* `options.path` scopes the search to a directory: its whole subtree by
* default, its immediate children when `recursive` is `false`. Both scopes
* are metadata filters the index SERVES, so the scope narrows the search
* before it runs no tree walk, and never an over-fetch filtered afterwards.
*
* @param query - The natural-language query.
* @param options - Scope, metadata filters and paging (see {@link SearchOptions}).
* @returns The matching files, best first.
* @throws {VFSError} ENOENT when `recursive: false` names a path that does
* not exist (the non-recursive scope is the directory's own identity, so
* the directory has to be there).
* Search files with natural language
*/
async search(query: string, options?: SearchOptions): Promise<SearchResult[]> {
await this.ensureInitialized()
@ -1600,26 +1588,11 @@ export class VirtualFileSystem implements IVirtualFileSystem {
}
}
// Scope to a directory, if asked. This used to emit
// `path: { $startsWith }` — an operator that is not in the filter
// vocabulary at all, and whose `$`-less spelling the metadata index
// REFUSES by the served-operator law (an equality/range posting index
// cannot evaluate a substring without reading every row). Every
// path-scoped VFS search therefore threw, and none has ever worked on
// this engine line. Both scopes below are served shapes.
// Add path filter if specified
if (options?.path) {
if (options.recursive === false) {
// Immediate children only: the directory's identity IS the scope, and
// `parent` is an indexed equality on every VFS entity.
params.where = {
...params.where,
parent: await this.pathResolver.resolve(options.path)
}
} else {
const scope = this.descendantPathScope(options.path)
if (scope) {
params.where = { ...params.where, path: scope }
}
params.where = {
...params.where,
path: { $startsWith: options.path }
}
}
@ -1781,42 +1754,6 @@ export class VirtualFileSystem implements IVirtualFileSystem {
return entity as VFSEntity
}
/**
* The SERVED metadata shape for "everything under this directory".
*
* `metadata.path` is the VFS's truth write and rename maintain it, and the
* `Contains` edges are a projection of it (see {@link repairContainment})
* it is indexed on every VFS entity, and the metadata index serves ordered
* range operators. So a subtree scope is a half-open range over the path
* column: O(log n + matches), no tree walk, and nothing fetched that the
* scope then discards.
*
* The range is `[dir + '/', dir + <successor of '/'>)`. Every descendant path
* begins with `dir + '/'`, and '0' is the code point directly after '/', so a
* string lies in the range EXACTLY when it carries that prefix. The two
* bounds differ at a single ASCII position, so the answer is the same under
* code-unit and code-point collation alike no dependence on how the store
* orders the rest of the string.
*
* Sibling exclusion falls out of the same fact and is worth stating, because
* it is where a naive prefix test goes wrong: for `dir = '/scope'`,
* `/scope-sibling/x` sorts BELOW the lower bound ('-' precedes '/') and
* `/scope0` sits at the open upper bound both outside, while
* `/scope/sub/deep/c.txt` is inside at any depth.
*
* @param path - The directory to scope to.
* @returns The `where` fragment for the `path` field, or `null` for the root
* every VFS entity is under it, so no clause narrows the search.
*/
private descendantPathScope(path: string): { gte: string; lt: string } | null {
const dir = path.replace(/\/+/g, '/').replace(/\/$/, '') || '/'
if (dir === '/') return null
// Computed, so the bound carries its own reason: the first string that can
// no longer share the `dir + '/'` prefix.
const separatorSuccessor = String.fromCharCode('/'.charCodeAt(0) + 1)
return { gte: `${dir}/`, lt: `${dir}${separatorSuccessor}` }
}
private getParentPath(path: string): string {
const normalized = path.replace(/\/+/g, '/').replace(/\/$/, '')
const lastSlash = normalized.lastIndexOf('/')

View file

@ -1,165 +0,0 @@
/**
* @module tests/vfs/vfs-search-path-scope
* @description `vfs.search({ path })` scopes with a SERVED filter.
*
* The scope used to be emitted as `path: { $startsWith }` an operator that is
* not in the filter vocabulary at all, and whose `$`-less spelling the metadata
* index refuses by the served-operator law (an equality/range posting index
* cannot evaluate a substring without reading every row). Every path-scoped VFS
* search threw; none has ever worked on this engine line.
*
* The scope is now a half-open range over `metadata.path`, which is the VFS's
* truth, is indexed on every VFS entity, and is served by the ordered range
* operators: `[dir + '/', dir + '0')` '0' being the code point after '/', so
* membership in the range is EXACTLY "carries the prefix `dir/`". The
* non-recursive scope is the directory's own identity, `parent`, an equality.
*
* These pins hold the answer (descendants at every depth, siblings never the
* `/scope-sibling` trap included), the shape (the operators the search emits
* are answered by the index's own door, never refused), and the law that the
* scope narrows the search BEFORE it runs rather than filtering an over-fetch.
*/
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest'
import { VirtualFileSystem } from '../../src/vfs/VirtualFileSystem.js'
import { Brainy } from '../../src/brainy.js'
import { VFSErrorCode } from '../../src/vfs/types.js'
/** A word every fixture file carries, so the text leg reaches all of them. */
const TOKEN = 'quasar'
describe('vfs.search({ path }) scopes with a served filter', () => {
let brain: Brainy
let vfs: VirtualFileSystem
/** In scope for '/scope', at three depths. */
const inScope = ['/scope/a.txt', '/scope/sub/b.txt', '/scope/sub/deep/c.txt']
/** Out of scope — including the two prefix traps a naive test misses. */
const outOfScope = ['/scope-sibling/d.txt', '/scope0/e.txt', '/elsewhere/f.txt', '/g.txt']
beforeAll(async () => {
brain = new Brainy({ requireSubtype: false, storage: { type: 'memory' }, silent: true })
await brain.init()
vfs = brain.vfs
await vfs.init()
await vfs.mkdir('/scope/sub/deep', { recursive: true })
await vfs.mkdir('/scope-sibling', { recursive: true })
await vfs.mkdir('/scope0', { recursive: true })
await vfs.mkdir('/elsewhere', { recursive: true })
for (const path of [...inScope, ...outOfScope]) {
await vfs.writeFile(path, `${TOKEN} content for ${path}`)
}
})
afterAll(async () => {
await vfs?.close()
await brain?.close()
})
it('includes every descendant depth and excludes every sibling', async () => {
const results = await vfs.search(TOKEN, { path: '/scope', limit: 50 })
const paths = results.map((r) => r.path).sort()
expect(paths).toEqual([...inScope].sort())
for (const path of outOfScope) expect(paths).not.toContain(path)
})
it('a trailing slash and a doubled slash name the same scope', async () => {
const plain = await vfs.search(TOKEN, { path: '/scope', limit: 50 })
const trailing = await vfs.search(TOKEN, { path: '/scope/', limit: 50 })
const doubled = await vfs.search(TOKEN, { path: '//scope//', limit: 50 })
const ids = (rs: Array<{ entityId: string }>) => rs.map((r) => r.entityId).sort()
expect(ids(trailing)).toEqual(ids(plain))
expect(ids(doubled)).toEqual(ids(plain))
})
it('the root scope is every VFS file — it adds no clause to narrow with', async () => {
const rooted = await vfs.search(TOKEN, { path: '/', limit: 50 })
const unscoped = await vfs.search(TOKEN, { limit: 50 })
const paths = rooted.map((r) => r.path).sort()
expect(paths).toEqual([...inScope, ...outOfScope].sort())
expect(paths).toEqual(unscoped.map((r) => r.path).sort())
})
it('recursive: false is the immediate children, not the subtree', async () => {
const results = await vfs.search(TOKEN, { path: '/scope', recursive: false, limit: 50 })
expect(results.map((r) => r.path)).toEqual(['/scope/a.txt'])
})
it('recursive: false on a path that does not exist refuses by name', async () => {
await expect(
vfs.search(TOKEN, { path: '/no-such-dir', recursive: false, limit: 50 })
).rejects.toMatchObject({ code: VFSErrorCode.ENOENT })
})
it('every operator the search emits is ANSWERED by the index door, never refused', async () => {
const index = (brain as any).metadataIndex
const emitted: any[] = []
const find = vi.spyOn(brain as any, 'find')
try {
await vfs.search(TOKEN, { path: '/scope', limit: 50 })
await vfs.search(TOKEN, { path: '/scope/sub', where: { mimeType: 'text/plain' }, limit: 50 })
await vfs.search(TOKEN, { path: '/scope', recursive: false, limit: 50 })
await vfs.search(TOKEN, { path: '/', limit: 50 })
for (const call of find.mock.calls) emitted.push((call[0] as any).where)
} finally {
find.mockRestore()
}
expect(emitted).toHaveLength(4)
for (const where of emitted) {
// The door itself is the judge: an operator outside the served set is
// REFUSED here (BrainyError INVALID_QUERY), never answered.
await expect(index.getIdsForFilter(where)).resolves.toBeInstanceOf(Array)
}
// And the scope really is a range on the path — the shape this fix chose.
expect(emitted[0].path).toEqual({ gte: '/scope/', lt: '/scope0' })
expect(emitted[3].path).toBeUndefined()
})
it('the scope narrows the search before it runs — no over-fetch to filter', async () => {
const index = (brain as any).metadataIndex
const filter = vi.spyOn(index, 'getIdsForFilter')
let universe: string[] = []
try {
await vfs.search(TOKEN, { path: '/scope', limit: 50 })
// The search's own call — the one carrying the scope. (Path resolution
// asks this same door for the root, before the search is built.)
const scoped = filter.mock.calls.findIndex(
(c) => (c[0] as any)?.path?.gte === '/scope/'
)
expect(scoped).toBeGreaterThanOrEqual(0)
universe = (await filter.mock.results[scoped].value) as string[]
} finally {
filter.mockRestore()
}
// The id universe the index resolved for the search is already the scope:
// three files, and not one row from outside it.
const rows = await brain.batchGet(universe)
const paths = [...rows.values()].map((e: any) => e.metadata.path).sort()
expect(paths).toEqual([...inScope].sort())
})
it('the range answers the same ids as walking the tree', async () => {
// The path is the truth and the Contains edges are its projection; a scope
// read from the truth must agree with one walked over the projection.
const walked: string[] = []
const walk = async (dir: string): Promise<void> => {
for (const name of await vfs.readdir(dir)) {
const child = dir === '/' ? `/${name}` : `${dir}/${name}`
const stat = await vfs.stat(child)
if (stat.isDirectory()) await walk(child)
else walked.push(child)
}
}
await walk('/scope')
const searched = await vfs.search(TOKEN, { path: '/scope', limit: 50 })
expect(searched.map((r) => r.path).sort()).toEqual(walked.sort())
})
})