feat(temporal): as-of semantic recall joins the release contract — past vectors byte-exact, pinned
The time-travel recall row moves from envelope-note to contracted: vector search at a pinned past generation serves the vectors AS THEY STOOD — a later re-embed never leaks into an earlier pin (byte-exact), tombstones mask, the deferred-embed pin serves the stub on the vector leg until the landing generation (text/metadata legs unaffected — triple intelligence by design), and beyond-head pins refuse typed. Brainy-alone leg = the documented ephemeral at-generation materialization; the at-scale leg rides the accelerated provider's as-of index. Registry row added (shared ID pending the master table).
This commit is contained in:
parent
13022c510b
commit
f7ca0d26de
2 changed files with 141 additions and 0 deletions
|
|
@ -43,6 +43,7 @@ and what's missing, stated) · 🔴 owed (named, never silent).
|
|||
| DP6 | Single write: ack at the canonical commit; visibility committed at ack (the atomic vector update kills the remove→add dark window); maintenance NEVER holds the ack (background flush cadence — THE ACK LAW pins: a hung flush cannot block a write, a hung EMBEDDER cannot block a write). | ✅ `tests/unit/brainy/persistence-policy` + `tests/unit/hnsw/update-item-atomic` + `tests/integration/deferred-embedding` |
|
||||
| DP7 | Bulk ingest: sustained rate holds flat — per-write maintenance taxes must not grow with brain size (A4 removed caller-flush convoys; deferred embedding removes the per-write embed tax where opted). | 🟡 the decay-curve row is a pair speed-table RED GATE; brainy-alone sustained-rate run rides the same corpora |
|
||||
| DP8 | Read under write pressure: no flicker window — a row that exists is never invisible to recall, even transiently (same-vector re-index is a no-op; changed-vector swaps in place, node never leaves the index; deferred updates serve the OLD vector until the atomic swap — stale-beats-absent). | ✅ brainy leg pinned (`tests/unit/hnsw/update-item-atomic` 9/9 + `deferred-embedding` stale-beats-absent); the symmetry property suite + runtime sentinels remain the B4 program |
|
||||
| — | **As-of semantic recall** (time-travel vector search): `asOf(G).find()` serves the vectors AS THEY STOOD at G — byte-exact past vectors, tombstone masking, the deferred-embed cell honest on the vector leg, TYPED refusal beyond the head. Brainy-alone leg = ephemeral at-generation materialization (documented O(n log n at G) build, bounded); the at-scale leg rides the accelerated provider's as-of index. | ✅ `tests/integration/asof-semantic-recall` 4/4 (registry ID pending the master table's mint) |
|
||||
| — | **The lazy-open gate honors EVERY provider's not-ready report** (a not-ready metadata provider can no longer latch the silent-empty state under `disableAutoRebuild`). | ✅ `tests/unit/brainy/lazy-notready-honor` |
|
||||
|
||||
## MT — Maintenance (never in the door path)
|
||||
|
|
|
|||
140
tests/integration/asof-semantic-recall.test.ts
Normal file
140
tests/integration/asof-semantic-recall.test.ts
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
/**
|
||||
* @module tests/integration/asof-semantic-recall
|
||||
* @description AS-OF SEMANTIC RECALL — the time-travel row of the release:
|
||||
* vector/semantic search at a pinned past generation, served EXACTLY.
|
||||
*
|
||||
* The contract pinned here (brainy-alone leg; the accelerated-provider leg
|
||||
* carries the same semantics at scale):
|
||||
* 1. PAST VECTORS ARE THE PAST'S VECTORS: a later re-embed/update never
|
||||
* leaks into an earlier pin — asOf(G) ranks by the vectors as they
|
||||
* stood at G, byte-exact.
|
||||
* 2. TOMBSTONE MASKING: a row deleted after G is FOUND at G; a row deleted
|
||||
* at or before G is ABSENT at G.
|
||||
* 3. THE DEFERRED-EMBED CELL of the visibility matrix: at pins before the
|
||||
* vector landed the row's VECTOR LEG serves the stub (text/metadata
|
||||
* legs may still surface it — triple intelligence by design); the real
|
||||
* vector serves only at and after its landing pin. No backward leak.
|
||||
* 4. TYPED REFUSAL beyond the log head — never a silent latest.
|
||||
*/
|
||||
import { describe, it, expect, afterEach } from 'vitest'
|
||||
import { Brainy } from '../../src/index.js'
|
||||
import { NounType } from '../../src/types/graphTypes.js'
|
||||
|
||||
const brains: Brainy[] = []
|
||||
|
||||
async function memBrain(): Promise<Brainy> {
|
||||
const b = new Brainy({ storage: { type: 'memory' }, requireSubtype: false })
|
||||
await b.init()
|
||||
brains.push(b)
|
||||
return b
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
for (const b of brains.splice(0)) await b.close().catch(() => {})
|
||||
})
|
||||
|
||||
describe('as-of semantic recall', () => {
|
||||
it('PAST VECTORS EXACT: a later update never leaks into an earlier pin', async () => {
|
||||
const brain = await memBrain()
|
||||
const id = await brain.add({
|
||||
data: 'crimson apples in the orchard',
|
||||
type: NounType.Document,
|
||||
metadata: { epoch: 'old' }
|
||||
})
|
||||
const g1 = brain.generation()
|
||||
const v1 = [...(((await brain.get(id, { includeVectors: true }))!.vector) as number[])]
|
||||
|
||||
await brain.update({ id, data: 'deep blue ocean currents', metadata: { epoch: 'new' } })
|
||||
const g2 = brain.generation()
|
||||
const v2 = (await brain.get(id, { includeVectors: true }))!.vector as number[]
|
||||
expect(v2, 'the update really re-embedded').not.toEqual(v1)
|
||||
|
||||
// The pin: at G1 the row carries its ORIGINAL vector and content.
|
||||
const dbPast = await brain.asOf(g1)
|
||||
const past = await dbPast.get(id, { includeVectors: true })
|
||||
expect(past, 'row exists at G1').toBeTruthy()
|
||||
expect(past!.vector as number[], 'as-of vector is byte-exact the OLD vector').toEqual(v1)
|
||||
expect((past!.metadata as { epoch: string }).epoch).toBe('old')
|
||||
|
||||
// Semantic search at G1 finds it via the OLD content; at G2 via the new.
|
||||
const hitsOld = await dbPast.find({ query: 'crimson apples in the orchard', limit: 3 })
|
||||
expect(hitsOld.map((r) => r.id), 'old content recalls at G1').toContain(id)
|
||||
const dbNow = await brain.asOf(g2)
|
||||
const hitsNew = await dbNow.find({ query: 'deep blue ocean currents', limit: 3 })
|
||||
expect(hitsNew.map((r) => r.id), 'new content recalls at G2').toContain(id)
|
||||
await dbPast.release()
|
||||
await dbNow.release()
|
||||
})
|
||||
|
||||
it('TOMBSTONE MASKING: deleted-after-G is found at G; deleted-before-G is absent', async () => {
|
||||
const brain = await memBrain()
|
||||
const doomed = await brain.add({
|
||||
data: 'ephemeral meteor shower observation',
|
||||
type: NounType.Document,
|
||||
metadata: {}
|
||||
})
|
||||
const keeper = await brain.add({
|
||||
data: 'permanent granite mountain survey',
|
||||
type: NounType.Document,
|
||||
metadata: {}
|
||||
})
|
||||
const gBoth = brain.generation()
|
||||
await brain.remove(doomed)
|
||||
const gAfter = brain.generation()
|
||||
|
||||
const dbBoth = await brain.asOf(gBoth)
|
||||
const atBoth = await dbBoth.find({ query: 'ephemeral meteor shower observation', limit: 5 })
|
||||
expect(atBoth.map((r) => r.id), 'pre-delete pin still recalls the row').toContain(doomed)
|
||||
|
||||
const dbAfter = await brain.asOf(gAfter)
|
||||
const atAfter = await dbAfter.find({ query: 'ephemeral meteor shower observation', limit: 5 })
|
||||
expect(atAfter.map((r) => r.id), 'post-delete pin masks the tombstoned row').not.toContain(doomed)
|
||||
expect((await dbAfter.find({ query: 'permanent granite mountain survey', limit: 5 })).map((r) => r.id)).toContain(keeper)
|
||||
await dbBoth.release()
|
||||
await dbAfter.release()
|
||||
})
|
||||
|
||||
it('DEFERRED-EMBED CELL: semantically absent before the vector landed, present after — never a stub match', async () => {
|
||||
const brain = await memBrain()
|
||||
// Anchor row so the semantic search always has a corpus.
|
||||
await brain.add({ data: 'unrelated anchor topic entirely', type: NounType.Document, metadata: {} })
|
||||
|
||||
const id = await brain.add({
|
||||
data: 'deferred saffron sunrise essay',
|
||||
type: NounType.Document,
|
||||
deferEmbedding: true,
|
||||
metadata: {}
|
||||
})
|
||||
const gAck = brain.generation()
|
||||
await brain.awaitPendingEmbeds()
|
||||
const gLanded = brain.generation()
|
||||
expect(gLanded, 'the landed vector is its own generation').toBeGreaterThan(gAck)
|
||||
|
||||
// At the ack generation: metadata-visible, and the VECTOR LEG carries
|
||||
// the stub (the visibility matrix's AT-EMBED cell governs the vector
|
||||
// leg — find({query})'s text/metadata legs may legitimately still
|
||||
// surface the row, that is triple intelligence working as designed;
|
||||
// what must NEVER happen is a stub vector ranking as a real one).
|
||||
const dbAck = await brain.asOf(gAck)
|
||||
const metaHits = await dbAck.find({ where: {}, limit: 10 })
|
||||
expect(metaHits.map((r) => r.id), 'metadata-visible at ack pin').toContain(id)
|
||||
const ackRow = await dbAck.get(id, { includeVectors: true })
|
||||
expect((ackRow!.vector as number[]).length, 'the as-of vector at the ack pin is the stub — no vector leaked backward').toBe(0)
|
||||
|
||||
// At the landed generation: fully recallable.
|
||||
const dbLanded = await brain.asOf(gLanded)
|
||||
const landedRow = await dbLanded.get(id, { includeVectors: true })
|
||||
expect((landedRow!.vector as number[]).length, 'the real vector serves at the landed pin').toBeGreaterThan(0)
|
||||
const semLanded = await dbLanded.find({ query: 'deferred saffron sunrise essay', limit: 5 })
|
||||
expect(semLanded.map((r) => r.id), 'recallable at the landed pin').toContain(id)
|
||||
await dbAck.release()
|
||||
await dbLanded.release()
|
||||
})
|
||||
|
||||
it('TYPED REFUSAL beyond the head — never a silent latest', async () => {
|
||||
const brain = await memBrain()
|
||||
await brain.add({ data: 'one row', type: NounType.Document, metadata: {} })
|
||||
const head = brain.generation()
|
||||
await expect(brain.asOf(head + 100)).rejects.toThrow(/generation|beyond|future|exceed/i)
|
||||
})
|
||||
})
|
||||
Loading…
Add table
Add a link
Reference in a new issue