feat(8.0): full query surface at historical generations via ephemeral index materialization
Historical Db values (now()/asOf() pins that history has moved past) now serve the COMPLETE query surface - vector/hybrid search, graph traversal, cursor pagination, and aggregation - by materializing ephemeral in-memory indexes over the exact at-generation record set. The historical-query throw is gone; NotYetSupportedAtHistoricalGenerationError is deleted. Materializer (Brainy.materializeAtGeneration): - Copies the at-G record set (live bytes for ids untouched since the pin, immutable before-images otherwise) into a fresh MemoryStorage; a final reconciliation pass under the commit mutex makes the copy exact even when transactions commit mid-build. - Opens a read-only Brainy over the copy: init rebuilds the metadata and graph-adjacency indexes from the records; the vector index is built by inserting every at-G vector (the at-G HNSW graph never existed on disk, so there is nothing to restore). Host embedder and aggregate definitions are shared - no second model load, aggregates backfill at-G values. - Cost is the documented contract: O(n at G) time and memory, ONCE per Db (handle cached; freed by release(), with a FinalizationRegistry backstop that also closes leaked readers). A native VersionedIndexProvider serves the same reads from retained segments with no rebuild. Db routing (src/db/db.ts): metadata-level find()/related() keep the free record path; index-only dimensions (query/vector/near/connected/cursor/ aggregate/includeRelations/non-metadata modes) route to the cached materialization; unsupported where-operators on the record path re-route there too instead of erroring. Speculative with() overlays keep the one honest boundary - SpeculativeOverlayError (overlay entities carry no embeddings, so index reads over them would be silently incomplete); metadata find()/get()/filter related() work on overlays. UpdateParams.vector contract now honored: an explicit pre-computed vector applies directly (with dimension validation) in update() and transact update ops, re-indexing HNSW - previously it was silently ignored unless data also changed. GraphAdjacencyIndex: adjacency now derives from the two verb-id LSM trees filtered through the live-verb tombstone set (entity->entity edge trees deleted - they carried no verb ids, so removeVerb could never tombstone them and traversal served stale neighbors forever). Neighbor reads batch- load live verbs via the unified cache; addVerb seeds the cache. Proofs (tests/integration/db-mvcc.test.ts, 24 green): historical vector search finds old vector placement including since-deleted entities; historical graph traversal walks the old wiring after a rewire; historical aggregation computes at-G group values; asOf() pins get the same surface; the materialization builds once per Db and release() closes the ephemeral reader (it refuses reads afterwards); overlays throw the documented error. ADR-001 updated to the no-throws historical model.
This commit is contained in:
parent
8f93add705
commit
e5feae4104
11 changed files with 869 additions and 340 deletions
|
|
@ -253,31 +253,44 @@ describe('Brainy.update()', () => {
|
|||
})).rejects.toThrow('invalid NounType')
|
||||
})
|
||||
|
||||
it('should not update vector directly via update method', async () => {
|
||||
it('applies an explicit pre-computed vector (UpdateParams.vector contract)', async () => {
|
||||
// Arrange
|
||||
const id = await brain.add(createAddParams({
|
||||
data: 'Test',
|
||||
type: 'thing'
|
||||
}))
|
||||
|
||||
// v5.11.1: Need includeVectors to get actual vector
|
||||
const original = await brain.get(id, { includeVectors: true })
|
||||
const originalVector = original!.vector
|
||||
|
||||
// Create a properly dimensioned but different vector
|
||||
const differentVector = originalVector.map(v => v * 2)
|
||||
|
||||
// Act - Update with vector param (not supported)
|
||||
// Act — update with an explicit vector and no new data. The
|
||||
// UpdateParams contract ("New pre-computed vector") applies it
|
||||
// directly, with no re-embedding (mirrored by transact update ops).
|
||||
await brain.update({
|
||||
id,
|
||||
vector: differentVector
|
||||
})
|
||||
|
||||
// Assert - Vector should not change (update ignores vector param)
|
||||
// v5.11.1: Need includeVectors to check vector
|
||||
// Assert — the stored vector is the supplied one.
|
||||
const updated = await brain.get(id, { includeVectors: true })
|
||||
expect(updated).not.toBeNull()
|
||||
expect(updated!.vector).toEqual(originalVector)
|
||||
expect(updated!.vector).toEqual(differentVector)
|
||||
})
|
||||
|
||||
it('rejects an explicit vector with mismatched dimensions', async () => {
|
||||
const id = await brain.add(createAddParams({
|
||||
data: 'Test',
|
||||
type: 'thing'
|
||||
}))
|
||||
|
||||
// Param validation rejects wrong dimensionality before the update runs
|
||||
// (update() also re-checks against the store's actual dimensionality).
|
||||
await expect(
|
||||
brain.update({ id, vector: [0.1, 0.2, 0.3] })
|
||||
).rejects.toThrow(/dimensions?/)
|
||||
})
|
||||
|
||||
it('should reject empty update parameters', async () => {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue