feat: portable graph export()/import() (BackupData v1) on brain.data()

brain.data() now serializes part or all of a brain to a versioned, portable JSON document (BackupData) and restores it — the path for partial backups, cross-environment moves, and 7.x→8.0 migration. - export(selector?, options?): select by ids, collection (+transitive Contains), connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural and predicate selectors compose. Options: includeVectors, includeContent (VFS blobs), includeSystem, edges ('induced'|'incident'|'none'). - import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip'; reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds to clone a subgraph under fresh ids. - BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/ relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence, weight, service) top-level; metadata custom-only. Current-state, no generations. - Replaces the prior flat-entity export() (dropped relations) with a graph-complete document. Distinct from brain.import(file) ingestion, which is unchanged. - Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/ ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export` now writes a BackupData document. Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00 · 2026-06-16 15:52:42 -07:00 · a408d3799d
commit a408d3799d
parent 89c6d043ba
11 changed files with 1393 additions and 382 deletions
--- a/docs/api/README.md
+++ b/docs/api/README.md
@ -1765,20 +1765,33 @@ await brain.import('https://api.example.com/data.json')

 ---

-### Export & Snapshots
+### Export & Import (portable backup)
+
+`brain.data()` exposes a portable graph backup/restore API. `export(selector?, options?)`
+serializes part or all of the graph to a versioned, portable `BackupData` document;
+`import(backup, options?)` restores it (dedup-by-id merge by default, re-embedding when
+vectors are absent).

 ```typescript
-// Export to file
-await brain.export('/path/to/backup.brainy')
+const data = await brain.data()

-// Create instant snapshot using COW fork
-await brain.fork('backup-2025-01-19')
+// Whole brain → a portable BackupData document
+const backup = await data.export()

-// Time-travel to specific commit
-const snapshot = await brain.asOf(commitId)
-const entities = await snapshot.find({ limit: 100 })
+// Just one workbench's members, with vectors, then restore elsewhere (merge by id)
+const subset = await data.export({ ids }, { includeVectors: true })
+await otherBrain.data().then(d => d.import(subset, { onConflict: 'merge' }))
+
+// A collection + its children; a connected neighbourhood; a VFS subtree (+ bytes)
+await data.export({ collection: collectionId })
+await data.export({ connected: { from: id, depth: 2 } })
+await data.export({ vfsPath: '/docs' }, { includeContent: true })
 ```

+The format is versioned (`formatVersion`) and current-state (no generation history) — see
+the **[Export & Import guide →](../guides/export-and-import.md)**. This is distinct from
+`brain.import(file)` (CSV/PDF/Excel/JSON ingestion).
+
 ---

 ## Configuration
--- a/docs/architecture/storage-architecture.md
+++ b/docs/architecture/storage-architecture.md
@ -457,23 +457,19 @@ await brain.storage.withLock('resource-id', async () => {

 ### Export Data
 ```typescript
-// Export entire database
-const backup = await brain.export({
- format: 'json',
- includeVectors: true,
- includeIndexes: false
-})
+// Export the whole brain to a portable BackupData document
+const backup = await brain.data().then(d => d.export(undefined, { includeVectors: true }))
 ```

 ### Import Data
 ```typescript
-// Import from backup
-await brain.import(backup, {
- mode: 'merge', // or 'replace'
- validateSchema: true
-})
+// Restore a BackupData document (dedup-by-id merge by default)
+await brain.data().then(d => d.import(backup, { onConflict: 'merge' }))
 ```

+See the [Export & Import guide](../guides/export-and-import.md) for partial exports
+(by id, collection, connected neighbourhood, VFS subtree, or predicate).
+
 ### Storage Migration
 ```typescript
 // Migrate between storage types
@ -483,9 +479,9 @@ const newBrain = new Brainy({ storage: { type: 's3' } })
 await oldBrain.init()
 await newBrain.init()

-// Transfer all data
-const data = await oldBrain.export()
-await newBrain.import(data)
+// Transfer all data via a portable backup
+const data = await oldBrain.data().then(d => d.export(undefined, { includeVectors: true }))
+await newBrain.data().then(d => d.import(data))
 ```

 ## Performance Tuning
--- a/docs/augmentations/EXAMPLES.md
+++ b/docs/augmentations/EXAMPLES.md
@ -546,11 +546,9 @@ const appComponent = await brain.findOne({ type: 'ReactComponent', name: 'App' }
 const imports = await brain.getRelated(appComponent.id, 'Imports')
 console.log(`App component imports:`, imports.map(c => c.name))

-// Export diagram
-const diagram = await brain.export({
-  format: 'react-diagram'
-})
-console.log(diagram.diagram) // Mermaid diagram
+// Export the analyzed component graph as a portable BackupData document
+const graph = await brain.data().then(d => d.export({ type: 'ReactComponent' }))
+console.log(`Exported ${graph.stats.entityCount} components, ${graph.stats.relationCount} edges`)
 ```

 ### Step 4: Premium Licensing (Optional)
--- a/docs/guides/export-and-import.md
+++ b/docs/guides/export-and-import.md
@ -0,0 +1,173 @@
+---
+title: Export & Import (portable graph)
+slug: guides/export-and-import
+public: true
+category: guides
+template: guide
+order: 9
+description: Export part or all of a brain to a portable, versioned BackupData document and import it back — by id, collection, connected neighbourhood, VFS subtree, predicate, or the whole brain. Covers vectors, edge policy, conflict handling, and id remapping.
+next:
+  - guides/subtypes-and-facets
+  - api/README
+---
+
+# Export & Import (portable graph)
+
+`brain.data()` exposes a **portable graph export/import** API. One method serializes a
+graph — an item, a collection, a connected neighbourhood, a VFS subtree, a predicate
+match, or the whole brain — into a single versioned JSON document (`BackupData`); the
+inverse restores it.
+
+```typescript
+const data = await brain.data()
+
+const backup = await data.export()          // whole brain → BackupData
+await data.import(backup)                    // restore (merge by id, re-embed if no vectors)
+```
+
+It is **portable** (human-readable JSON), **versioned** (`formatVersion`, so a 7.x export
+imports cleanly into 8.0), and **current-state** (the entities and edges as they are now —
+no generation history). Use it for portable artifacts, partial backups, cross-environment
+moves, and version upgrades.
+
+## When to use which
+
+| You want… | Use |
+|-----------|-----|
+| A portable, partial-or-whole, cross-version graph document | **`brain.data().export()` / `import()`** (this guide) |
+| To ingest a CSV / PDF / Excel / JSON **file** as new entities | `brain.import(file)` — see [Import Anything](./import-anything.md) |
+
+The two are different operations that happen to share a verb: `import(file)` parses a
+foreign document into new entities; `data().import(backup)` restores a graph this API
+exported.
+
+## Exporting
+
+```typescript
+export(selector?, options?): Promise<BackupData>
+```
+
+### Selectors — *what* to export
+
+Omit the selector to export the whole brain. Otherwise pick a node set:
+
+| Scenario | Selector |
+|----------|----------|
+| Just an item (or items) | `{ ids: ['a', 'b'] }` |
+| A collection + its children | `{ collection: collectionId }` (alias `memberOf`) |
+| A connected neighbourhood | `{ connected: { from: id, depth: 2, verbs?, direction? } }` |
+| A VFS directory / file (+ subtree) | `{ vfsPath: '/docs', recursive?: true }` |
+| Everything matching a predicate | `{ type, subtype, where, service }` |
+| The whole brain | *(omit)* |
+
+The selector reuses `find()`'s grammar — *"export what `find()` would match, minus ranking
+and limit."* Structural and predicate selectors **compose** — a structural selector picks
+the nodes, and predicate keys then filter them:
+
+```typescript
+// Members of a collection whose status is "open"
+await data.export({ collection: collectionId, where: { status: 'open' } })
+```
+
+If you already have results from `find()`, export exactly those with the `ids` selector:
+
+```typescript
+const hits = await brain.find({ type: NounType.Document })
+await data.export({ ids: hits.map(r => r.id) })
+```
+
+### Options — *how* to serialize
+
+| Option | Default | Effect |
+|--------|---------|--------|
+| `includeVectors` | `false` | Carry embedding vectors verbatim. Off ⇒ `import()` re-embeds from `data`. |
+| `includeContent` | `false` | Include VFS file bytes in `blobs` so files round-trip byte-identically. |
+| `includeSystem` | `false` | Include `system` entities such as the VFS root. |
+| `edges` | `'induced'` | `'induced'` (both endpoints in the set), `'incident'` (also dangling edges, recorded in `danglingIds`), or `'none'` (nodes only). |
+
+```typescript
+// A self-contained subgraph with vectors, ready to restore elsewhere
+const subset = await data.export({ ids }, { includeVectors: true })
+
+// A VFS subtree including file bytes
+const tree = await data.export({ vfsPath: '/docs' }, { includeContent: true })
+```
+
+## Importing
+
+```typescript
+import(backup, options?): Promise<ImportResult>
+```
+
+```typescript
+const result = await brain.data().then(d => d.import(backup, { onConflict: 'merge' }))
+// → { imported, merged, skipped, reembedded, blobsWritten, errors }
+```
+
+| Option | Default | Effect |
+|--------|---------|--------|
+| `onConflict` | `'merge'` | `'merge'` (update existing id in place — assemble many backups), `'replace'` (delete + recreate), or `'skip'`. |
+| `reembed` | `'auto'` | `'auto'` (use the carried vector, else re-embed from `data`) or `'never'` (require a carried vector; record an error if absent). |
+| `remapIds` | — | Rewrite every id on the way in, e.g. to clone a template subgraph under fresh ids. |
+
+The default `onConflict: 'merge'` is what lets you assemble one working graph from many
+exported documents that share entity ids — re-importing an id merges rather than duplicates.
+
+```typescript
+// Clone a subgraph under fresh ids (a copy, not a move)
+const remap = new Map(backup.entities.map(e => [e.id, crypto.randomUUID()]))
+await data.import(backup, { remapIds: id => remap.get(id) ?? id })
+```
+
+## The `BackupData` format
+
+```jsonc
+{
+  "format": "brainy-backup",
+  "formatVersion": 1,                 // import gates on this (cross-version migration)
+  "brainyVersion": "7.32.0",
+  "createdAt": "2026-06-16T…Z",
+  "embedding": { "model": "all-MiniLM-L6-v2", "dimensions": 384 },
+  "selector": { … },                  // echoes what was exported (provenance)
+  "entities": [
+    {
+      "id": "…", "type": "Document", "subtype": "invoice",
+      "data": "…",                    // the embedding source
+      "confidence": 1, "weight": 1, "service": "…",
+      "vector": [ … ],                // only with includeVectors
+      "metadata": { … }               // custom fields only (reserved fields are top-level)
+    }
+  ],
+  "relations": [
+    { "id":"…", "from":"…", "to":"…", "type":"Contains", "subtype":"…",
+      "weight":1, "confidence":1, "metadata": { … } }
+  ],
+  "blobs": { "<sha256>": "<base64>" }, // only with includeContent
+  "danglingIds": [ "…" ],             // only with edges:'incident'
+  "stats": { "entityCount": 0, "relationCount": 0, "blobCount": 0, "vectorDimensions": 384 }
+}
+```
+
+Standard fields (`subtype`, `data`, `confidence`, `weight`, `service`) sit at the top level
+of each entity; `metadata` holds **only** custom user fields — mirroring the in-memory
+`Entity` shape, so `import()` maps each field to its dedicated parameter.
+
+## Cross-version (7.x → 8.0)
+
+Because the document is shared and versioned, a backup written by 7.x imports into 8.0:
+`formatVersion` is read forward, `subtype` is carried so 8.0 re-types correctly, and the
+same 384-dimension model on both lines means `includeVectors:false` re-embeds identically
+(or `true` carries vectors verbatim). The format is **current-state** — if you need a
+whole-brain snapshot *with* generation history, that is a separate native facility
+(`db.persist()` / `Brainy.load()` on 8.0).
+
+## VFS
+
+VFS directories are `Collection` entities and files are entities linked by `Contains`, so
+the whole filesystem (or any subtree) exports through the `vfsPath` selector:
+
+```typescript
+await data.export({ vfsPath: '/' },        { includeContent: true }) // all VFS + bytes
+await data.export({ vfsPath: '/docs' },    { includeContent: true }) // one directory
+await data.export({ vfsPath: '/a/b.txt' }, { includeContent: true }) // one file
+```
--- a/docs/guides/framework-integration.md
+++ b/docs/guides/framework-integration.md
@ -540,11 +540,11 @@ export async function generateStaticProps() {
  })
  await brain.init()

-  // Build search index
-  const allContent = await brain.export()
+  // Build a search index — export the whole brain as a portable BackupData document
+  const backup = await brain.data().then(d => d.export())

  return {
-    props: { searchIndex: allContent }
+    props: { searchIndex: backup.entities }
  }
 }
 ```