From c2b73d456450702371b84e6d6383870651e90ab1 Mon Sep 17 00:00:00 2001 From: David Snelling Date: Wed, 17 Jun 2026 11:24:57 -0700 Subject: [PATCH] docs(8.0): export/import guide + api/README portable backup section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - New docs/guides/export-and-import.md (public) for the 8.0 surface: brain.export()/import(), Db composition (asOf/with time-travel + what-if export), selectors, options, BackupData v1 format, cross-version (7.x→8.0), VFS, and the generations/persist distinction. Documents only the implemented surface. - api/README "Export & Import (portable) + Snapshots (native)": adds the portable brain.export()/import() round-trip alongside the native persist()/asOf() snapshot. --- docs/api/README.md | 27 ++++- docs/guides/export-and-import.md | 181 +++++++++++++++++++++++++++++++ 2 files changed, 207 insertions(+), 1 deletion(-) create mode 100644 docs/guides/export-and-import.md diff --git a/docs/api/README.md b/docs/api/README.md index e4b2b55c..4a9dc3f6 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -1413,7 +1413,32 @@ await brain.import('https://api.example.com/data.json') --- -### Export & Snapshots +### Export & Import (portable) + Snapshots (native) + +**Portable graph backup** — `brain.export()` / `brain.import()` (`BackupData` v1, versioned +JSON, partial-or-whole, cross-version). `export()` lives on the immutable `Db`, so it composes +with `now()`/`asOf()`/`with()`: + +```typescript +// Export part or all of the brain to a portable, versioned document +const backup = await brain.export({ ids }, { includeVectors: true }) + +// Restore it — import() routes a BackupData to the graph round-trip (merge by id) +await otherBrain.import(backup, { onConflict: 'merge' }) + +// Time-travel export (serialize a past generation) / what-if export (a speculative state) +const past = await brain.asOf(gen) +const asWas = await past.export({ collection: id }) +await past.release() +await brain.now().with(ops).export({ ids }) +``` + +Selectors: `{ ids }`, `{ collection }` (alias `memberOf`), `{ connected: { from, depth } }`, +`{ vfsPath }`, predicate (`{ type, subtype, where, service }`), or whole brain (omit). See the +**[Export & Import guide](../guides/export-and-import.md)**. Distinct from `brain.import(file)` +(CSV/PDF/Excel/JSON ingestion — `import()` dispatches on whether you pass a `BackupData` or a file). + +**Native whole-brain snapshot** (generation-preserving, not portable JSON): ```typescript // Instant hard-link snapshot via the Db API diff --git a/docs/guides/export-and-import.md b/docs/guides/export-and-import.md new file mode 100644 index 00000000..45f3a47d --- /dev/null +++ b/docs/guides/export-and-import.md @@ -0,0 +1,181 @@ +--- +title: Export & Import (portable graph) +slug: guides/export-and-import +public: true +category: guides +template: guide +order: 9 +description: Export part or all of a brain to a portable, versioned BackupData document and import it back — by id, collection, connected neighbourhood, VFS subtree, predicate, or the whole brain. export() lives on the immutable Db, so asOf()/with() give time-travel and what-if exports. +next: + - guides/subtypes-and-facets + - api/README +--- + +# Export & Import (portable graph) + +Brainy serializes part or all of a brain — an item, a collection, a connected +neighbourhood, a VFS subtree, a predicate match, or the whole brain — into a single +versioned JSON document (`BackupData`), and restores it. + +```typescript +const backup = await brain.export() // whole brain → BackupData +await brain.import(backup) // restore (merge by id, re-embed if no vectors) +``` + +It is **portable** (human-readable JSON), **versioned** (`formatVersion`, so a document +written by 7.x imports cleanly into 8.0), and **current-state** (the entities and edges as +they are now — no generation history). Use it for portable artifacts, partial backups, +cross-environment moves, and version upgrades. + +`export()` is a method on the **immutable `Db` value**, so it composes with every way of +obtaining one: + +```typescript +brain.export(sel) // = brain.now().export(sel) +;(await brain.asOf(gen)).export(sel) // time-travel export (a past generation) +brain.now().with(ops).export(sel) // what-if export (a speculative state) +``` + +## When to use which + +| You want… | Use | +|-----------|-----| +| A portable, partial-or-whole, cross-version graph document | **`brain.export()` / `brain.import()`** (this guide) | +| A whole-brain snapshot **with generation history** | `brain.now().persist(path)` / `Brainy.load(path)` (native) | +| To ingest a CSV / PDF / Excel / JSON **file** as new entities | `brain.import(file)` — see [Import Anything](./import-anything.md) | + +`import()` is **polymorphic**: hand it a `BackupData` and it does the graph round-trip; +hand it a file/buffer and it does foreign-file ingestion (dispatched on the document's +`format: 'brainy-backup'` tag). + +## Exporting + +```typescript +brain.export(selector?, options?): Promise +// (also on any Db: brain.now().export(...), (await brain.asOf(g)).export(...)) +``` + +### Selectors — *what* to export + +Omit the selector to export the whole brain. Otherwise pick a node set: + +| Scenario | Selector | +|----------|----------| +| Just an item (or items) | `{ ids: ['a', 'b'] }` | +| A collection + its children | `{ collection: collectionId }` (alias `memberOf`) | +| A connected neighbourhood | `{ connected: { from: id, depth: 2, verbs?, direction? } }` | +| A VFS directory / file (+ subtree) | `{ vfsPath: '/docs', recursive?: true }` | +| Everything matching a predicate | `{ type, subtype, where, service, visibility }` | +| The whole brain | *(omit)* | + +The selector reuses `find()`'s grammar — *"export what `find()` would match, minus ranking +and limit."* Structural and predicate selectors **compose**: + +```typescript +// Members of a collection whose status is "open" +await brain.export({ collection: collectionId, where: { status: 'open' } }) + +// Already have find() results? Export exactly those with the ids selector +const hits = await brain.find({ type: NounType.Document }) +await brain.export({ ids: hits.map(r => r.id) }) +``` + +### Options — *how* to serialize + +| Option | Default | Effect | +|--------|---------|--------| +| `includeVectors` | `false` | Carry embedding vectors verbatim. Off ⇒ `import()` re-embeds from `data`. | +| `includeContent` | `false` | Include VFS file bytes in `blobs` so files round-trip byte-identically. | +| `includeSystem` | `false` | Include `visibility:'system'` entities such as the VFS root. | +| `edges` | `'induced'` | `'induced'` (both endpoints in the set), `'incident'` (also dangling edges, recorded in `danglingIds`), or `'none'` (nodes only). | + +## Importing + +```typescript +brain.import(backup, options?): Promise +``` + +The whole backup is applied as **one atomic transaction** — it advances the brain exactly +one generation, or none on failure. + +```typescript +const result = await brain.import(backup, { onConflict: 'merge' }) +// → { imported, merged, skipped, reembedded, blobsWritten, errors } +``` + +| Option | Default | Effect | +|--------|---------|--------| +| `onConflict` | `'merge'` | `'merge'` (update existing id in place — assemble many backups), `'replace'` (delete + recreate), or `'skip'`. | +| `reembed` | `'auto'` | `'auto'` (use the carried vector, else re-embed from `data`) or `'never'` (require a carried vector; record an error if absent). | +| `remapIds` | — | Rewrite every id on the way in, e.g. to clone a template subgraph under fresh ids. | +| `meta` | — | Transaction metadata recorded in the tx-log alongside the new generation. | + +The default `onConflict: 'merge'` lets you assemble one working graph from many exported +documents that share entity ids — re-importing an id merges rather than duplicates. + +## The `BackupData` format + +```jsonc +{ + "format": "brainy-backup", + "formatVersion": 1, // import gates on this (cross-version migration) + "brainyVersion": "8.0.0", + "createdAt": "2026-06-16T…Z", + "embedding": { "model": "all-MiniLM-L6-v2", "dimensions": 384 }, + "selector": { … }, // echoes what was exported (provenance) + "entities": [ + { + "id": "…", "type": "Document", "subtype": "invoice", "visibility": "public", + "data": "…", // the embedding source + "confidence": 1, "weight": 1, "service": "…", + "vector": [ … ], // only with includeVectors + "metadata": { … } // custom fields only (reserved fields are top-level) + } + ], + "relations": [ + { "id":"…", "from":"…", "to":"…", "type":"Contains", "subtype":"…", + "weight":1, "confidence":1, "metadata": { … } } + ], + "blobs": { "": "" }, // only with includeContent + "danglingIds": [ "…" ], // only with edges:'incident' + "stats": { "entityCount": 0, "relationCount": 0, "blobCount": 0, "vectorDimensions": 384 } +} +``` + +Standard fields (`subtype`, `visibility`, `data`, `confidence`, `weight`, `service`) sit at +the top level of each entity; `metadata` holds **only** custom user fields — mirroring the +in-memory `Entity` shape, so `import()` maps each field to its dedicated parameter. The +TypeScript types (`BackupData`, `BackupEntity`, `BackupRelation`, `ExportSelector`, +`ExportOptions`, `ImportOptions`, `ImportResult`) are exported from the package root. + +## Generations & time-travel + +The portable document is **current-state** — it never embeds generation history (that keeps +it cross-version-portable). History lives where it's queryable: + +- **During a session:** `brain.asOf(g)` / `brain.now().with(ops)` on the live brain. Because + `export()` is on the `Db`, `(await brain.asOf(g)).export()` serializes a *past* generation + and `brain.now().with(ops).export()` serializes a *speculative* one. +- **A whole-brain snapshot with history:** `brain.now().persist(path)` / `Brainy.load(path)` + (native, generation-preserving) — a separate facility from this portable format. + +Note: only `transact()` (and the write shortcuts that commit through it) advances a +generation, so time-travel export differs across transaction boundaries. + +## Cross-version (7.x → 8.0) + +Because the document is shared and versioned, a backup written by 7.x imports into 8.0: +`formatVersion` is read forward, `subtype` is carried so 8.0 re-types correctly, and the +same 384-dimension model on both lines means `includeVectors:false` re-embeds identically +(or `true` carries vectors verbatim). + +## VFS + +VFS directories are `Collection` entities and files are entities linked by `Contains`, so +the whole filesystem (or any subtree) exports through the `vfsPath` selector: + +```typescript +await brain.export({ vfsPath: '/' }, { includeContent: true }) // all VFS + bytes +await brain.export({ vfsPath: '/docs' }, { includeContent: true }) // one directory +await brain.export({ vfsPath: '/a/b.txt' }, { includeContent: true }) // one file +```