feat(8.0): generational MVCC storage + Datomic-style Db API (now/transact/asOf/with/persist)
One mechanism replaces the COW and versioning subsystems: immutable
generation-stamped records behind a Datomic-style database value.
Record layer (src/db/generationStore.ts):
- Monotonic generation counter in _system/generation.json; bumped once per
transact() commit and once per single-operation write (storage hook), so
brain.generation() is always a meaningful watermark.
- Commit protocol: stage before-images + tx.json delta -> fsync -> execute
batch via TransactionManager -> atomic tmp+rename of _system/manifest.json
(the rename IS the commit point) -> append _system/tx-log.jsonl.
- Crash recovery on open rolls uncommitted generations back byte-identically
and forces an index rebuild; refcounted pins gate compactHistory(), which
records a horizon (asOf below it throws GenerationCompactedError).
Db API:
- Db: get/find/search/related pinned at a generation, with() speculative
overlays, since() diffs, persist() hard-link-farm snapshots, timestamp,
generation, release() + FinalizationRegistry backstop.
- Brainy: now() O(1) pin, transact(ops, {meta, ifAtGeneration}) atomic batch
(GenerationConflictError CAS), asOf(generation|Date|path), restore(path,
{confirm}), compactHistory(), generation(), static open() + load().
- get()/metadata find()/related() are fully correct at any reachable pinned
generation; index-accelerated queries at historical generations throw
NotYetSupportedAtHistoricalGenerationError - never silently-wrong results.
- VersionedIndexProvider (generation/isGenerationVisible/pin/release) in
plugin.ts: feature-detected, balanced pin/release in lockstep with Db
lifecycle, post-commit applier + replay-gap model documented.
Storage primitives (BaseStorage + filesystem/memory adapters): raw-object
read/write/list/remove, fsync barrier, noun/verb raw before-image capture,
tx-log append, snapshotToDirectory (hard-link farm; byte-copy fallback and
append-in-place exceptions), restoreFromDirectory + derived-state reload.
Proof suite (tests/integration/db-mvcc.test.ts + tests/unit/db/): isolation
across 200 mutations, batch atomicity under injected execution failure,
ifAtGeneration CAS, snapshot immunity to source mutation, compaction safety
under pins, with() overlay isolation, generation monotonicity across
reopen, crash consistency through the real recovery path, and versioned
provider pin/release balance. Design record in
docs/ADR-001-generational-mvcc.md.
This commit is contained in:
parent
49e49483d1
commit
431cd64406
16 changed files with 6244 additions and 5 deletions
806
src/db/db.ts
Normal file
806
src/db/db.ts
Normal file
|
|
@ -0,0 +1,806 @@
|
|||
/**
|
||||
* @module db/db
|
||||
* @description `Db` — Brainy 8.0's immutable, Datomic-style database value.
|
||||
*
|
||||
* A `Db` is a **readonly view pinned at one generation** of a Brainy store.
|
||||
* It is produced by `brain.now()` (O(1) pin of the current generation),
|
||||
* `brain.transact()` (pinned at the freshly committed generation),
|
||||
* `brain.asOf()` (a past generation, a timestamp, or a persisted snapshot
|
||||
* path), `Brainy.load(path)` (a snapshot opened read-only), and `db.with()`
|
||||
* (a speculative in-memory overlay).
|
||||
*
|
||||
* **Read semantics.** While no transaction has committed past the pinned
|
||||
* generation, every read delegates straight to the live brain — the pinned
|
||||
* view IS the current state, and the existing fast paths serve it untouched.
|
||||
* Once later transactions commit, the `Db` resolves reads through the
|
||||
* generational record layer: `get()` and metadata-level `find()`/`related()`
|
||||
* remain fully correct at the pinned generation (changed ids are resolved
|
||||
* from immutable before-images; unchanged ids still ride the live fast
|
||||
* path), while index-accelerated queries (semantic/vector search) throw the
|
||||
* documented {@link NotYetSupportedAtHistoricalGenerationError} — an honest
|
||||
* boundary, never silently-wrong results.
|
||||
*
|
||||
* **History granularity.** Generation records are written per `transact()`
|
||||
* batch. Single-operation writes (`add`/`update`/`relate`/… outside
|
||||
* `transact()`) advance the generation counter but do not produce historical
|
||||
* records, so they remain visible through earlier pins — the documented 8.0
|
||||
* granularity (see `docs/ADR-001-generational-mvcc.md`, "History
|
||||
* granularity").
|
||||
*
|
||||
* **Lifecycle.** Each `Db` holds one refcounted pin on its generation (and
|
||||
* on every registered {@link ../plugin.js VersionedIndexProvider}). Call
|
||||
* {@link Db.release} when done — a `FinalizationRegistry` backstop releases
|
||||
* leaked pins on garbage collection, but explicit release is what makes
|
||||
* `compactHistory()` deterministic.
|
||||
*/
|
||||
|
||||
import type {
|
||||
Entity,
|
||||
FindParams,
|
||||
GetOptions,
|
||||
GetRelationsParams,
|
||||
Relation,
|
||||
Result
|
||||
} from '../types/brainy.types.js'
|
||||
import { v4 as uuidv4 } from '../universal/uuid.js'
|
||||
import { NotYetSupportedAtHistoricalGenerationError } from './errors.js'
|
||||
import type { GenerationStore } from './generationStore.js'
|
||||
import type { ChangedIds, TransactReceipt, TxOperation } from './types.js'
|
||||
import { entityMatchesFind, resolveEntityField, UnsupportedWhereOperatorError } from './whereMatcher.js'
|
||||
|
||||
/**
|
||||
* @description The speculative overlay carried by a `db.with()` value:
|
||||
* per-id replacement entities/relations (`null` = tombstone). Reads on the
|
||||
* overlay `Db` consult these maps first, then fall through to the pinned
|
||||
* base view. Overlays are pure in-memory values — they never touch disk or
|
||||
* index providers.
|
||||
*/
|
||||
export interface SpeculativeOverlay<T = any> {
|
||||
/** Entity overlays: replacement entity, or `null` for a speculative delete. */
|
||||
nouns: Map<string, Entity<T> | null>
|
||||
/** Relation overlays: replacement relation, or `null` for a speculative delete. */
|
||||
verbs: Map<string, Relation<T> | null>
|
||||
}
|
||||
|
||||
/**
|
||||
* @description The host surface a `Db` needs from its `Brainy` instance —
|
||||
* constructed once per brain (see `brainy.ts`). Internal: consumers never
|
||||
* build one; `Db` values are only created by `Brainy` and `db.with()`.
|
||||
*/
|
||||
export interface DbHost<T = any> {
|
||||
/** The generational record layer of the host brain. */
|
||||
readonly store: GenerationStore
|
||||
/** Live `brain.get()` (current-generation fast path). */
|
||||
get(id: string, options?: GetOptions): Promise<Entity<T> | null>
|
||||
/** Live `brain.find()` (current-generation fast path). */
|
||||
find(query: string | FindParams<T>): Promise<Result<T>[]>
|
||||
/** Live `brain.getRelations()` (current-generation fast path). */
|
||||
getRelations(paramsOrId?: string | GetRelationsParams): Promise<Relation<T>[]>
|
||||
/** Materialize an entity from a generation record's raw stored objects. */
|
||||
entityFromRecord(
|
||||
id: string,
|
||||
record: { metadata: any; vector: any | null },
|
||||
includeVectors: boolean
|
||||
): Promise<Entity<T>>
|
||||
/** Materialize a relation from a generation record's raw stored objects. */
|
||||
relationFromRecord(id: string, record: { metadata: any; vector: any | null }): Relation<T> | null
|
||||
/** Snapshot the store at `generation` into `targetPath` (throws if the pin is stale). */
|
||||
persistPinned(targetPath: string, generation: number): Promise<void>
|
||||
/** Add one refcounted pin (store + versioned providers) on `generation`. */
|
||||
pinGeneration(generation: number): void
|
||||
/** Release one refcounted pin (store + versioned providers) on `generation`. */
|
||||
releaseGeneration(generation: number): void
|
||||
/** Register a `Db` with the GC-backstop finalization registry. */
|
||||
registerDbForFinalization(db: Db<T>, generation: number, closeOnRelease?: () => Promise<void>): void
|
||||
/** Unregister a `Db` after an explicit release (no double-release on GC). */
|
||||
unregisterDbFromFinalization(db: Db<T>): void
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Internal constructor arguments for {@link Db}. The pin on
|
||||
* `generation` is taken by the creator (`Brainy` or `db.with()`) **before**
|
||||
* construction; the constructor only registers the GC backstop.
|
||||
*/
|
||||
export interface DbInit<T = any> {
|
||||
/** The host surface (one per brain). */
|
||||
host: DbHost<T>
|
||||
/** The pinned generation. */
|
||||
generation: number
|
||||
/** The view's timestamp (ms since epoch) — see {@link Db.timestamp}. */
|
||||
timestamp: number
|
||||
/** Per-operation receipt, present only on `transact()`-produced values. */
|
||||
receipt?: TransactReceipt
|
||||
/** Speculative overlay, present only on `with()`-produced values. */
|
||||
overlay?: SpeculativeOverlay<T>
|
||||
/** Extra teardown on release (e.g. closing an owned snapshot brain). */
|
||||
closeOnRelease?: () => Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* @description An immutable, generation-pinned view of a Brainy store. See
|
||||
* the module documentation for the full read-semantics contract.
|
||||
*
|
||||
* @example
|
||||
* const db = brain.now() // O(1) pin
|
||||
* await brain.transact([{ op: 'update', id, metadata: { v: 2 } }])
|
||||
* await db.get(id) // still sees v: 1 — pinned
|
||||
* await brain.get(id) // sees v: 2 — live
|
||||
* await db.release() // unpin (enables compaction)
|
||||
*/
|
||||
export class Db<T = any> {
|
||||
private readonly host: DbHost<T>
|
||||
private readonly gen: number
|
||||
private readonly ts: number
|
||||
private readonly overlay?: SpeculativeOverlay<T>
|
||||
private readonly closeOnRelease?: () => Promise<void>
|
||||
private isReleased = false
|
||||
|
||||
/**
|
||||
* Per-operation receipt (committed generation, timestamp, resolved ids in
|
||||
* input order). Present only on values returned by `brain.transact()`.
|
||||
*/
|
||||
public readonly receipt?: TransactReceipt
|
||||
|
||||
/**
|
||||
* @param init - Internal construction arguments. `Db` values are created
|
||||
* by `Brainy` (`now`/`transact`/`asOf`/`load`) and `db.with()` — never
|
||||
* directly by consumers.
|
||||
*/
|
||||
constructor(init: DbInit<T>) {
|
||||
this.host = init.host
|
||||
this.gen = init.generation
|
||||
this.ts = init.timestamp
|
||||
this.receipt = init.receipt
|
||||
this.overlay = init.overlay
|
||||
this.closeOnRelease = init.closeOnRelease
|
||||
this.host.registerDbForFinalization(this, this.gen, this.closeOnRelease)
|
||||
}
|
||||
|
||||
/** The generation this view is pinned at. */
|
||||
get generation(): number {
|
||||
return this.gen
|
||||
}
|
||||
|
||||
/**
|
||||
* The view's wall-clock timestamp (ms since epoch): pin time for `now()`,
|
||||
* commit time for `transact()`, and the resolved commit time for `asOf()`.
|
||||
*/
|
||||
get timestamp(): number {
|
||||
return this.ts
|
||||
}
|
||||
|
||||
/** Whether {@link Db.release} has been called (released views throw on use). */
|
||||
get released(): boolean {
|
||||
return this.isReleased
|
||||
}
|
||||
|
||||
/** Whether this view carries a speculative `with()` overlay. */
|
||||
get speculative(): boolean {
|
||||
return this.overlay !== undefined
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Reads
|
||||
// ==========================================================================
|
||||
|
||||
/**
|
||||
* @description Get an entity by id **as of this view's generation**.
|
||||
* Overlay entries win (speculative views); ids untouched since the pin ride
|
||||
* the live fast path; ids changed by later transactions resolve from
|
||||
* immutable generation records. Always correct at the pinned generation.
|
||||
*
|
||||
* @param id - The entity id.
|
||||
* @param options - Same options as `brain.get()` (`includeVectors`).
|
||||
* @returns The entity as of this generation, or `null`.
|
||||
*/
|
||||
async get(id: string, options?: GetOptions): Promise<Entity<T> | null> {
|
||||
this.assertUsable('get')
|
||||
|
||||
if (this.overlay && this.overlay.nouns.has(id)) {
|
||||
return this.overlay.nouns.get(id) ?? null
|
||||
}
|
||||
|
||||
if (!this.isHistorical()) {
|
||||
return this.host.get(id, options)
|
||||
}
|
||||
|
||||
const resolved = await this.host.store.resolveAt('noun', id, this.gen)
|
||||
if (resolved.source === 'current') {
|
||||
return this.host.get(id, options)
|
||||
}
|
||||
if (resolved.source === 'absent' || resolved.metadata === null) {
|
||||
// Live semantics: an entity without metadata is not a live entity.
|
||||
return null
|
||||
}
|
||||
return this.host.entityFromRecord(
|
||||
id,
|
||||
{ metadata: resolved.metadata, vector: resolved.vector },
|
||||
options?.includeVectors ?? false
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Metadata-level `find()` **as of this view's generation**.
|
||||
*
|
||||
* At the current generation (nothing committed past the pin, no overlay)
|
||||
* this delegates to `brain.find()` untouched — the full query surface,
|
||||
* including semantic search. At a historical generation, or on a
|
||||
* speculative view, only the metadata dimensions are supported (`type`,
|
||||
* `subtype`, `where`, `service`, `excludeVFS`, `orderBy`/`order`,
|
||||
* `limit`/`offset`): results are computed by combining the live index for
|
||||
* unchanged ids with per-entity evaluation of the same filters against
|
||||
* generation records / overlay entries — set-correct at the pinned
|
||||
* generation. Index-only dimensions (`query`, `vector`, `near`,
|
||||
* `connected`, `cursor`, `aggregate`, …) throw
|
||||
* {@link NotYetSupportedAtHistoricalGenerationError}.
|
||||
*
|
||||
* Result ordering is deterministic when `orderBy` is supplied; without it,
|
||||
* overlay/record-resolved matches follow the live-index matches.
|
||||
*
|
||||
* @param query - A `FindParams` object, or a string (semantic search —
|
||||
* current-generation views only).
|
||||
* @returns Matching results as of this generation (score `1.0` for
|
||||
* record-resolved metadata matches, mirroring live metadata-only finds).
|
||||
*/
|
||||
async find(query: string | FindParams<T>): Promise<Result<T>[]> {
|
||||
this.assertUsable('find')
|
||||
|
||||
const params: FindParams<T> = typeof query === 'string' ? { query } : query
|
||||
const historical = this.isHistorical()
|
||||
|
||||
if (!historical && !this.overlay) {
|
||||
return this.host.find(query)
|
||||
}
|
||||
|
||||
this.assertMetadataOnlyFind(params)
|
||||
|
||||
const limit = params.limit ?? 10
|
||||
const offset = params.offset ?? 0
|
||||
|
||||
// Ids whose live-index answer is NOT valid at this generation.
|
||||
const changedNouns = historical
|
||||
? (await this.host.store.changedBetween(this.gen, this.host.store.committedGeneration())).nouns
|
||||
: []
|
||||
const overlayNouns = this.overlay?.nouns ?? new Map<string, Entity<T> | null>()
|
||||
const excluded = new Set<string>([...changedNouns, ...overlayNouns.keys()])
|
||||
|
||||
try {
|
||||
// Over-fetch from the live index so dropping superseded ids cannot
|
||||
// starve the requested page, then merge and re-window.
|
||||
const base = await this.host.find({
|
||||
...params,
|
||||
limit: limit + offset + excluded.size,
|
||||
offset: 0
|
||||
})
|
||||
const merged = base.filter((result) => !excluded.has(result.id))
|
||||
|
||||
for (const id of changedNouns) {
|
||||
if (overlayNouns.has(id)) continue // The overlay supersedes history.
|
||||
const resolved = await this.host.store.resolveAt('noun', id, this.gen)
|
||||
let entity: Entity<T> | null = null
|
||||
if (resolved.source === 'record' && resolved.metadata !== null) {
|
||||
entity = await this.host.entityFromRecord(
|
||||
id,
|
||||
{ metadata: resolved.metadata, vector: resolved.vector },
|
||||
false
|
||||
)
|
||||
} else if (resolved.source === 'current') {
|
||||
// Defensive: changed ids always resolve to a record or absent, but
|
||||
// a 'current' answer is still served correctly from the live path.
|
||||
entity = await this.host.get(id)
|
||||
}
|
||||
if (entity && entityMatchesFind(entity as Entity, params as FindParams)) {
|
||||
merged.push(resultFromEntity(entity))
|
||||
}
|
||||
}
|
||||
|
||||
for (const [, entity] of overlayNouns) {
|
||||
if (entity === null) continue // Speculative delete.
|
||||
if (entityMatchesFind(entity as Entity, params as FindParams)) {
|
||||
merged.push(resultFromEntity(entity))
|
||||
}
|
||||
}
|
||||
|
||||
if (params.orderBy) {
|
||||
sortResultsBy(merged, params.orderBy, params.order ?? 'asc')
|
||||
}
|
||||
return merged.slice(offset, offset + limit)
|
||||
} catch (err) {
|
||||
if (err instanceof UnsupportedWhereOperatorError) {
|
||||
throw new NotYetSupportedAtHistoricalGenerationError(
|
||||
`metadata find with where-operator '${err.operator}'`,
|
||||
this.gen,
|
||||
this.host.store.generation()
|
||||
)
|
||||
}
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Semantic (vector) search **at the current generation only**.
|
||||
* Convenience for `find({ query, ...options })`. Vector indexes serve the
|
||||
* live state; at a historical generation, or on a speculative overlay
|
||||
* (whose entities carry no embeddings), this throws the documented
|
||||
* {@link NotYetSupportedAtHistoricalGenerationError} — persist the
|
||||
* generation you need and `Brainy.load()` it for the full query surface.
|
||||
*
|
||||
* @param query - Natural-language search query.
|
||||
* @param options - Additional `FindParams` (limit, filters, …).
|
||||
* @returns Semantic search results (current-generation views only).
|
||||
* @throws NotYetSupportedAtHistoricalGenerationError at historical
|
||||
* generations and on speculative overlays.
|
||||
*/
|
||||
async search(query: string, options?: Omit<FindParams<T>, 'query'>): Promise<Result<T>[]> {
|
||||
this.assertUsable('search')
|
||||
|
||||
if (!this.isHistorical() && !this.overlay) {
|
||||
return this.host.find({ ...(options ?? {}), query })
|
||||
}
|
||||
throw new NotYetSupportedAtHistoricalGenerationError(
|
||||
this.overlay ? 'vector search (speculative overlay)' : 'vector search',
|
||||
this.gen,
|
||||
this.host.store.generation()
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Relationships **as of this view's generation** — the `Db`
|
||||
* counterpart of `brain.getRelations()` (same string-id shorthand and
|
||||
* filter surface). Relations whose edges changed after the pin are
|
||||
* resolved from generation records; overlay relations (speculative
|
||||
* `relate`/`unrelate`) and cascade tombstones (relations touching a
|
||||
* speculatively deleted entity) are applied on top. Cursor pagination is
|
||||
* index-only and throws the documented historical error on
|
||||
* non-current/speculative views.
|
||||
*
|
||||
* @param paramsOrId - An entity id (shorthand for `{ from: id }`) or a
|
||||
* `GetRelationsParams` filter object.
|
||||
* @returns Relations as of this generation.
|
||||
*/
|
||||
async related(paramsOrId?: string | GetRelationsParams): Promise<Relation<T>[]> {
|
||||
this.assertUsable('related')
|
||||
|
||||
const params: GetRelationsParams =
|
||||
typeof paramsOrId === 'string' ? { from: paramsOrId } : (paramsOrId ?? {})
|
||||
const historical = this.isHistorical()
|
||||
|
||||
if (!historical && !this.overlay) {
|
||||
return this.host.getRelations(paramsOrId)
|
||||
}
|
||||
|
||||
if (params.cursor !== undefined) {
|
||||
throw new NotYetSupportedAtHistoricalGenerationError(
|
||||
'cursor pagination on related()',
|
||||
this.gen,
|
||||
this.host.store.generation()
|
||||
)
|
||||
}
|
||||
|
||||
const limit = params.limit ?? 100
|
||||
const offset = params.offset ?? 0
|
||||
|
||||
const changedVerbs = historical
|
||||
? (await this.host.store.changedBetween(this.gen, this.host.store.committedGeneration())).verbs
|
||||
: []
|
||||
const overlayVerbs = this.overlay?.verbs ?? new Map<string, Relation<T> | null>()
|
||||
const excluded = new Set<string>([...changedVerbs, ...overlayVerbs.keys()])
|
||||
|
||||
const base = await this.host.getRelations({
|
||||
...params,
|
||||
limit: limit + offset + excluded.size,
|
||||
offset: 0
|
||||
})
|
||||
let merged = base.filter((relation) => !excluded.has(relation.id))
|
||||
|
||||
for (const id of changedVerbs) {
|
||||
if (overlayVerbs.has(id)) continue
|
||||
const resolved = await this.host.store.resolveAt('verb', id, this.gen)
|
||||
if (resolved.source !== 'record') continue
|
||||
const relation = this.host.relationFromRecord(id, {
|
||||
metadata: resolved.metadata,
|
||||
vector: resolved.vector
|
||||
})
|
||||
if (relation && relationMatchesParams(relation, params)) {
|
||||
merged.push(relation)
|
||||
}
|
||||
}
|
||||
|
||||
for (const [, relation] of overlayVerbs) {
|
||||
if (relation === null) continue // Speculative unrelate.
|
||||
if (relationMatchesParams(relation, params)) {
|
||||
merged.push(relation)
|
||||
}
|
||||
}
|
||||
|
||||
// Cascade semantics for speculative deletes: a relation whose endpoint
|
||||
// is tombstoned in the overlay is gone in this view, exactly as
|
||||
// brain.delete() cascades on the durable path.
|
||||
if (this.overlay) {
|
||||
const tombstoned = new Set<string>()
|
||||
for (const [id, entity] of this.overlay.nouns) {
|
||||
if (entity === null) tombstoned.add(id)
|
||||
}
|
||||
if (tombstoned.size > 0) {
|
||||
merged = merged.filter(
|
||||
(relation) => !tombstoned.has(relation.from) && !tombstoned.has(relation.to)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
return merged.slice(offset, offset + limit)
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Derived views
|
||||
// ==========================================================================
|
||||
|
||||
/**
|
||||
* @description Speculative write: returns a NEW `Db` whose reads see the
|
||||
* given transaction data applied **in memory, on top of this view** —
|
||||
* Datomic's `with`. Nothing touches disk, the generation counter, or index
|
||||
* providers; the underlying brain and this view are untouched. Use it to
|
||||
* answer "what would the store look like if…" questions, then call
|
||||
* `brain.transact()` with the same operations to make it real.
|
||||
*
|
||||
* Speculative semantics (documented deltas from the durable paths):
|
||||
* - `add` without an explicit `vector` produces an entity with an empty
|
||||
* stub vector — speculative entities are not semantically searchable
|
||||
* (`with()` never invokes the embedder).
|
||||
* - `remove` cascades on the read side: relations touching the removed
|
||||
* entity disappear from `related()` without being enumerated.
|
||||
* - `relate` deduplicates against edges visible in this view when verb
|
||||
* history allows the check, mirroring `brain.relate()`.
|
||||
* - Brain-level vocabulary/subtype enforcement does not run — overlays
|
||||
* never persist, so there is nothing to protect.
|
||||
*
|
||||
* The returned view takes its own pin on this generation; release both
|
||||
* views independently.
|
||||
*
|
||||
* @param ops - The same declarative operations `brain.transact()` accepts.
|
||||
* @returns A new speculative `Db` over this view.
|
||||
* @throws Error when an operation references a missing entity/relation,
|
||||
* mirroring the durable path's planning errors.
|
||||
*/
|
||||
async with(ops: TxOperation<T>[]): Promise<Db<T>> {
|
||||
this.assertUsable('with')
|
||||
if (!Array.isArray(ops) || ops.length === 0) {
|
||||
throw new Error('with(): provide a non-empty array of transaction operations')
|
||||
}
|
||||
|
||||
// Child overlay starts as a copy of this view's overlay (chained with()).
|
||||
const overlay: SpeculativeOverlay<T> = {
|
||||
nouns: new Map(this.overlay?.nouns ?? []),
|
||||
verbs: new Map(this.overlay?.verbs ?? [])
|
||||
}
|
||||
|
||||
// Reads during planning must see "this view + overlay so far".
|
||||
const speculativeGet = async (id: string): Promise<Entity<T> | null> => {
|
||||
if (overlay.nouns.has(id)) return overlay.nouns.get(id) ?? null
|
||||
return this.get(id)
|
||||
}
|
||||
|
||||
for (const op of ops) {
|
||||
switch (op.op) {
|
||||
case 'add': {
|
||||
const id = op.id ?? uuidv4()
|
||||
const now = Date.now()
|
||||
overlay.nouns.set(id, {
|
||||
id,
|
||||
vector: op.vector ?? [],
|
||||
type: op.type,
|
||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
||||
data: op.data,
|
||||
metadata: (op.metadata ?? {}) as T,
|
||||
...(op.service !== undefined && { service: op.service }),
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
...(op.confidence !== undefined && { confidence: op.confidence }),
|
||||
...(op.weight !== undefined && { weight: op.weight }),
|
||||
_rev: 1
|
||||
})
|
||||
break
|
||||
}
|
||||
case 'update': {
|
||||
const base = await speculativeGet(op.id)
|
||||
if (!base) {
|
||||
throw new Error(`with(): entity ${op.id} not found at generation ${this.gen}`)
|
||||
}
|
||||
const mergedMetadata =
|
||||
op.merge !== false
|
||||
? ({ ...(base.metadata as object), ...(op.metadata as object) } as T)
|
||||
: ((op.metadata ?? base.metadata) as T)
|
||||
overlay.nouns.set(op.id, {
|
||||
...base,
|
||||
...(op.type !== undefined && { type: op.type }),
|
||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
||||
...(op.data !== undefined && { data: op.data }),
|
||||
...(op.vector !== undefined && { vector: op.vector }),
|
||||
...(op.confidence !== undefined && { confidence: op.confidence }),
|
||||
...(op.weight !== undefined && { weight: op.weight }),
|
||||
metadata: mergedMetadata,
|
||||
updatedAt: Date.now(),
|
||||
_rev: (base._rev ?? 1) + 1
|
||||
})
|
||||
break
|
||||
}
|
||||
case 'remove': {
|
||||
overlay.nouns.set(op.id, null)
|
||||
// Tombstone overlay relations touching the removed entity; base
|
||||
// relations are cascade-filtered at read time in related().
|
||||
for (const [verbId, relation] of overlay.verbs) {
|
||||
if (relation && (relation.from === op.id || relation.to === op.id)) {
|
||||
overlay.verbs.set(verbId, null)
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'relate': {
|
||||
const from = await speculativeGet(op.from)
|
||||
const to = await speculativeGet(op.to)
|
||||
if (!from) throw new Error(`with(): source entity ${op.from} not found`)
|
||||
if (!to) throw new Error(`with(): target entity ${op.to} not found`)
|
||||
|
||||
// Dedupe against the view (overlay first, then committed edges
|
||||
// when verb history allows the read) — mirror of relate().
|
||||
let duplicate: Relation<T> | undefined
|
||||
for (const relation of overlay.verbs.values()) {
|
||||
if (relation && relation.from === op.from && relation.to === op.to && relation.type === op.type) {
|
||||
duplicate = relation
|
||||
break
|
||||
}
|
||||
}
|
||||
if (!duplicate) {
|
||||
try {
|
||||
const existing = await this.related({ from: op.from, type: op.type })
|
||||
duplicate = existing.find((relation) => relation.to === op.to)
|
||||
} catch (err) {
|
||||
if (!(err instanceof NotYetSupportedAtHistoricalGenerationError)) throw err
|
||||
// Verb history can't serve the check at this generation —
|
||||
// proceed with overlay-only dedupe (documented).
|
||||
}
|
||||
}
|
||||
if (duplicate) break
|
||||
|
||||
const id = uuidv4()
|
||||
overlay.verbs.set(id, {
|
||||
id,
|
||||
from: op.from,
|
||||
to: op.to,
|
||||
type: op.type,
|
||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
||||
weight: op.weight ?? 1.0,
|
||||
...(op.data !== undefined && { data: op.data }),
|
||||
metadata: op.metadata,
|
||||
...(op.service !== undefined && { service: op.service }),
|
||||
createdAt: Date.now()
|
||||
})
|
||||
if (op.bidirectional) {
|
||||
const reverseId = uuidv4()
|
||||
overlay.verbs.set(reverseId, {
|
||||
id: reverseId,
|
||||
from: op.to,
|
||||
to: op.from,
|
||||
type: op.type,
|
||||
...(op.subtype !== undefined && { subtype: op.subtype }),
|
||||
weight: op.weight ?? 1.0,
|
||||
...(op.data !== undefined && { data: op.data }),
|
||||
metadata: op.metadata,
|
||||
...(op.service !== undefined && { service: op.service }),
|
||||
createdAt: Date.now()
|
||||
})
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'unrelate': {
|
||||
overlay.verbs.set(op.id, null)
|
||||
break
|
||||
}
|
||||
default: {
|
||||
const exhaustive: never = op
|
||||
throw new Error(`with(): unknown operation ${JSON.stringify(exhaustive)}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
this.host.pinGeneration(this.gen)
|
||||
return new Db<T>({
|
||||
host: this.host,
|
||||
generation: this.gen,
|
||||
timestamp: Date.now(),
|
||||
overlay
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* @description The ids changed by committed transactions in the interval
|
||||
* `(priorDb.generation, this.generation]` — the diff between two pinned
|
||||
* views of the same store. Single-operation writes between commits are not
|
||||
* reflected (documented 8.0 history granularity).
|
||||
*
|
||||
* @param priorDb - An earlier view of the SAME store.
|
||||
* @returns Changed entity and relation ids, sorted.
|
||||
* @throws RangeError when `priorDb` is newer than this view, or Error when
|
||||
* the views belong to different stores.
|
||||
*/
|
||||
async since(priorDb: Db<T>): Promise<ChangedIds> {
|
||||
this.assertUsable('since')
|
||||
if (priorDb.host.store !== this.host.store) {
|
||||
throw new Error('since(): both Db values must come from the same Brainy instance')
|
||||
}
|
||||
if (priorDb.gen > this.gen) {
|
||||
throw new RangeError(
|
||||
`since(): priorDb is at generation ${priorDb.gen}, which is newer than this view ` +
|
||||
`(generation ${this.gen}). Call newerDb.since(olderDb).`
|
||||
)
|
||||
}
|
||||
return this.host.store.changedBetween(priorDb.gen, this.gen)
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Durability
|
||||
// ==========================================================================
|
||||
|
||||
/**
|
||||
* @description Snapshot this view to `path` as a self-contained store:
|
||||
* flush, then hard-link every immutable file into the target
|
||||
* (Cassandra-style — instant and space-shared; cross-device targets fall
|
||||
* back to byte copies, and the small append-in-place file list is always
|
||||
* byte-copied). The result opens with `Brainy.load(path)` with the full
|
||||
* query surface, and later mutations of the source can never alter it —
|
||||
* rewrites swap inodes, the snapshot keeps the old bytes.
|
||||
*
|
||||
* `persist()` requires this view to still be the store's **latest**
|
||||
* generation (snapshotting captures current bytes): pin with `brain.now()`
|
||||
* and persist before further writes. A view that history has moved past
|
||||
* throws {@link NotYetSupportedAtHistoricalGenerationError}; speculative
|
||||
* overlays cannot be persisted (commit them with `brain.transact()` first).
|
||||
*
|
||||
* @param path - Absolute directory for the snapshot (created; must be
|
||||
* empty or absent).
|
||||
* @throws NotYetSupportedAtHistoricalGenerationError when this view is no
|
||||
* longer the latest generation, or is speculative.
|
||||
*/
|
||||
async persist(path: string): Promise<void> {
|
||||
this.assertUsable('persist')
|
||||
if (this.overlay) {
|
||||
throw new NotYetSupportedAtHistoricalGenerationError(
|
||||
'persist of a speculative overlay',
|
||||
this.gen,
|
||||
this.host.store.generation()
|
||||
)
|
||||
}
|
||||
await this.host.persistPinned(path, this.gen)
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Lifecycle
|
||||
// ==========================================================================
|
||||
|
||||
/**
|
||||
* @description Release this view's pin (store + versioned index
|
||||
* providers). Idempotent. After release, every read throws. Views from
|
||||
* `Brainy.load()` / `asOf(path)` also close their underlying read-only
|
||||
* brain here. A `FinalizationRegistry` backstop releases leaked pins when
|
||||
* a `Db` is garbage-collected, but explicit release is what makes
|
||||
* `compactHistory()` deterministic — prefer it.
|
||||
*/
|
||||
async release(): Promise<void> {
|
||||
if (this.isReleased) return
|
||||
this.isReleased = true
|
||||
this.host.unregisterDbFromFinalization(this)
|
||||
this.host.releaseGeneration(this.gen)
|
||||
if (this.closeOnRelease) {
|
||||
await this.closeOnRelease()
|
||||
}
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Internals
|
||||
// ==========================================================================
|
||||
|
||||
/** Whether any transaction committed past the pinned generation. */
|
||||
private isHistorical(): boolean {
|
||||
return this.host.store.hasCommittedAfter(this.gen)
|
||||
}
|
||||
|
||||
/** Throw on use-after-release. */
|
||||
private assertUsable(method: string): void {
|
||||
if (this.isReleased) {
|
||||
throw new Error(
|
||||
`Cannot call ${method}() on a released Db (generation ${this.gen}). ` +
|
||||
`Pin a fresh view with brain.now() or brain.asOf().`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject find() dimensions that require index machinery the record layer
|
||||
* cannot answer at a historical generation / for a speculative overlay.
|
||||
*/
|
||||
private assertMetadataOnlyFind(params: FindParams<T>): void {
|
||||
const unsupported: Array<[string, boolean]> = [
|
||||
['semantic query', params.query !== undefined],
|
||||
['vector search', params.vector !== undefined],
|
||||
['proximity search (near)', params.near !== undefined],
|
||||
['graph traversal (connected)', params.connected !== undefined],
|
||||
['cursor pagination', params.cursor !== undefined],
|
||||
['aggregation', params.aggregate !== undefined],
|
||||
['relation expansion (includeRelations)', params.includeRelations === true],
|
||||
[
|
||||
`search mode '${params.mode ?? params.searchMode}'`,
|
||||
(params.mode !== undefined && params.mode !== 'metadata' && params.mode !== 'auto') ||
|
||||
(params.searchMode !== undefined && params.searchMode !== 'auto')
|
||||
]
|
||||
]
|
||||
for (const [capability, present] of unsupported) {
|
||||
if (present) {
|
||||
throw new NotYetSupportedAtHistoricalGenerationError(
|
||||
capability,
|
||||
this.gen,
|
||||
this.host.store.generation()
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Build a `Result` row from a record/overlay-resolved entity,
|
||||
* mirroring the live metadata-only find path (flattened fields, score `1.0`).
|
||||
*/
|
||||
function resultFromEntity<T>(entity: Entity<T>): Result<T> {
|
||||
return {
|
||||
id: entity.id,
|
||||
score: 1.0,
|
||||
type: entity.type,
|
||||
subtype: entity.subtype,
|
||||
metadata: entity.metadata,
|
||||
data: entity.data,
|
||||
confidence: entity.confidence,
|
||||
weight: entity.weight,
|
||||
_rev: entity._rev,
|
||||
entity
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @description In-place stable sort of merged results by a find() `orderBy`
|
||||
* field, mirroring live ordering semantics (`asc` default, undefined values
|
||||
* last in both directions).
|
||||
*/
|
||||
function sortResultsBy<T>(results: Result<T>[], orderBy: string, order: 'asc' | 'desc'): void {
|
||||
const direction = order === 'desc' ? -1 : 1
|
||||
results.sort((a, b) => {
|
||||
const va = resolveEntityField(a.entity as Entity, orderBy)
|
||||
const vb = resolveEntityField(b.entity as Entity, orderBy)
|
||||
if (va === undefined && vb === undefined) return 0
|
||||
if (va === undefined) return 1
|
||||
if (vb === undefined) return -1
|
||||
if (typeof va === 'number' && typeof vb === 'number') return (va - vb) * direction
|
||||
const sa = String(va)
|
||||
const sb = String(vb)
|
||||
return (sa < sb ? -1 : sa > sb ? 1 : 0) * direction
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Filter one relation against `GetRelationsParams`, mirroring
|
||||
* the storage-level filter `brain.getRelations()` builds (`from` → sourceId,
|
||||
* `to` → targetId, type/subtype set membership, service equality).
|
||||
*/
|
||||
function relationMatchesParams<T>(relation: Relation<T>, params: GetRelationsParams): boolean {
|
||||
if (params.from !== undefined && relation.from !== params.from) return false
|
||||
if (params.to !== undefined && relation.to !== params.to) return false
|
||||
if (params.type !== undefined) {
|
||||
const types = Array.isArray(params.type) ? params.type : [params.type]
|
||||
if (!types.includes(relation.type)) return false
|
||||
}
|
||||
if (params.subtype !== undefined) {
|
||||
const subtypes = Array.isArray(params.subtype) ? params.subtype : [params.subtype]
|
||||
if (relation.subtype === undefined || !subtypes.includes(relation.subtype)) return false
|
||||
}
|
||||
if (params.service !== undefined && relation.service !== params.service) return false
|
||||
return true
|
||||
}
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue