feat(8.0): portable graph export()/import() (BackupData v1) — Db.export + polymorphic import
Re-adds the portable graph round-trip on 8.0 (deleted with DataAPI), now on the immutable Db so it composes with the generational model. - src/db/backup.ts: BackupData v1 engine (identical wire format to the 7.32.0 line). exportGraph() reads through a Db at its pinned generation; importGraph() applies the whole backup as ONE atomic transact() (a single generation). - Db.export(selector?, options?) — read at this view's generation, so brain.asOf(g).export() is a time-travel export and brain.now().with(ops).export() a what-if export. - brain.export(selector?, options?) — sugar for now().export(). - brain.import() is polymorphic: a BackupData document -> graph round-trip; a file/buffer -> existing foreign-file ingestion (dispatched on the 'brainy-backup' tag; zero migration for ingestion callers). - Selectors (ids/collection/connected/vfsPath/predicate/whole) + edge policy (induced/incident/none) + includeVectors/includeContent/includeSystem; system entities excluded by visibility. onConflict merge/replace/skip, reembed auto/never, remapIds. DbHost gains storage (VFS blob bytes). - Types exported from the package root: BackupData/BackupEntity/BackupRelation/ ExportSelector/ExportOptions/ImportOptions/ImportResult + isBackupData. Tests: tests/unit/db/db-backup.test.ts (10, green) — round-trip, selectors, edges, vectors+re-embed, merge, asOf/with composition, validation. Full unit suite green. Follow-ups: docs (8.0 guide + RELEASES + api/README), completeness adds (since(prior).export() delta, exportStream/importStream, validate()), includeContent filesystem test, ids-at-asOf get() refinement, @soulcraft/formats Zod schema mirror.
This commit is contained in:
parent
f4dea80176
commit
010ccf816d
5 changed files with 948 additions and 4 deletions
|
|
@ -115,6 +115,15 @@ import type { AggregateDefinition, AggregateQueryParams, AggregateResult } from
|
|||
import { resolveJsHnswConfig, DEFAULT_RECALL } from './utils/recallPreset.js'
|
||||
import * as fs from 'node:fs'
|
||||
import { Db, type DbHost, type HistoricalQueryHandle } from './db/db.js'
|
||||
import {
|
||||
importGraph,
|
||||
isBackupData,
|
||||
type BackupData,
|
||||
type ExportSelector,
|
||||
type ExportOptions,
|
||||
type ImportOptions,
|
||||
type ImportResult
|
||||
} from './db/backup.js'
|
||||
import { GenerationStore } from './db/generationStore.js'
|
||||
import { isDeterministicEmbedMode } from './embeddings/deterministicEmbedMode.js'
|
||||
import { GenerationConflictError } from './db/errors.js'
|
||||
|
|
@ -5060,6 +5069,26 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Serialize part or all of the brain into a portable `BackupData`
|
||||
* document — sugar for `brain.now().export(selector, options)` (the current
|
||||
* generation). For a past generation use `(await brain.asOf(g)).export(...)`;
|
||||
* for a speculative state use `brain.now().with(ops).export(...)`.
|
||||
*
|
||||
* Restore a `BackupData` with {@link Brainy.import} (it dispatches a backup to the
|
||||
* graph round-trip; a file/buffer to ingestion). Distinct from `db.persist()`
|
||||
* (native whole-brain snapshot with generation history).
|
||||
*
|
||||
* @param selector - WHAT to export (omit for the whole brain). See {@link ExportSelector}.
|
||||
* @param options - HOW to export (vectors / VFS bytes / edge policy). See {@link ExportOptions}.
|
||||
* @returns A versioned, portable `BackupData` document.
|
||||
* @example
|
||||
* const backup = await brain.export({ ids }, { includeVectors: true })
|
||||
*/
|
||||
async export(selector: ExportSelector = {}, options: ExportOptions = {}): Promise<BackupData> {
|
||||
return this.now().export(selector, options)
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Execute a declarative operation batch atomically: either
|
||||
* every operation applies and the store advances exactly ONE generation,
|
||||
|
|
@ -5434,6 +5463,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
if (!this._dbHost) {
|
||||
this._dbHost = {
|
||||
store: this.generationStore,
|
||||
storage: this.storage,
|
||||
get: (id, options) => this.get(id, options),
|
||||
find: (query) => this.find(query),
|
||||
related: (paramsOrId) => this.related(paramsOrId),
|
||||
|
|
@ -6709,7 +6739,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
* - Reduced confusion (removed redundant options)
|
||||
*/
|
||||
async import(
|
||||
source: Buffer | string | object,
|
||||
source: Buffer | string | object | BackupData,
|
||||
options?: {
|
||||
format?: 'excel' | 'pdf' | 'csv' | 'json' | 'markdown' | 'yaml' | 'docx' | 'image'
|
||||
vfsPath?: string
|
||||
|
|
@ -6734,14 +6764,20 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
throughput?: number
|
||||
eta?: number
|
||||
}) => void
|
||||
} & Partial<ImportOptions>
|
||||
): Promise<ImportResult | any> {
|
||||
// Portable graph round-trip: a BackupData document (produced by export()) is
|
||||
// restored via ONE atomic transaction — distinct from foreign-file ingestion below.
|
||||
if (isBackupData(source)) {
|
||||
return importGraph(this, this.storage, source, options as ImportOptions)
|
||||
}
|
||||
) {
|
||||
// Lazy load ImportCoordinator
|
||||
|
||||
// Lazy load ImportCoordinator (foreign-file ingestion: CSV/PDF/Excel/JSON/…)
|
||||
const { ImportCoordinator } = await import('./import/ImportCoordinator.js')
|
||||
const coordinator = new ImportCoordinator(this)
|
||||
await coordinator.init()
|
||||
|
||||
return await coordinator.import(source, options)
|
||||
return await coordinator.import(source as Buffer | string | object, options)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue