brainy/docs/guides/export-and-import.md
David Snelling 89036deb20 refactor: rename BackupData → PortableGraph (the type is interchange, not a backup)
Parity with the 8.0 rename, backported to the 7.x line. The brain.data()
export()/import() document type was BackupData, but it is the portable, versioned
interchange representation of a graph (entities + relations + optional vectors),
NOT a backup — exported for transport between instances, versions, and products.
The actual backup is the native snapshot, so "Backup*" mis-signalled.

Rename every developer-visible symbol, JSDoc, comment and doc:
- BackupData→PortableGraph, BackupEntity→PortableGraphEntity,
  BackupRelation→PortableGraphRelation, BACKUP_FORMAT[_VERSION]→
  PORTABLE_GRAPH_FORMAT[_VERSION], internal toBackup* helpers→toPortableGraph*.
- src/api/DataAPI.ts, src/index.ts, src/cli/commands/core.ts, docs, and the test
  (renamed data-backup.test.ts → data-portable-graph.test.ts).

The on-the-wire `format` tag is also renamed 'brainy-backup' → 'brainy-portable-graph':
the export/import format was introduced in 7.32.0 and has not been adopted by any
consumer, so there are no stored documents to stay compatible with — a clean rename
beats carrying a legacy tag forward. No deprecated aliases (nothing to alias).

1505 unit green; build clean; data-portable-graph.test.ts 20/20.
2026-06-19 12:30:06 -07:00

7.2 KiB

title slug public category template order description next
Export & Import (portable graph) guides/export-and-import true guides guide 9 Export part or all of a brain to a portable, versioned PortableGraph document and import it back — by id, collection, connected neighbourhood, VFS subtree, predicate, or the whole brain. Covers vectors, edge policy, conflict handling, and id remapping.
guides/subtypes-and-facets
api/README

Export & Import (portable graph)

brain.data() exposes a portable graph export/import API. One method serializes a graph — an item, a collection, a connected neighbourhood, a VFS subtree, a predicate match, or the whole brain — into a single versioned JSON document (PortableGraph); the inverse restores it.

const data = await brain.data()

const graph = await data.export()          // whole brain → PortableGraph
await data.import(graph)                    // restore (merge by id, re-embed if no vectors)

It is portable (human-readable JSON), versioned (formatVersion, so a 7.x export imports cleanly into 8.0), and current-state (the entities and edges as they are now — no generation history). Use it for portable artifacts, partial graphs, cross-environment moves, and version upgrades.

When to use which

You want… Use
A portable, partial-or-whole, cross-version graph document brain.data().export() / import() (this guide)
To ingest a CSV / PDF / Excel / JSON file as new entities brain.import(file) — see Import Anything

The two are different operations that happen to share a verb: import(file) parses a foreign document into new entities; data().import(graph) restores a graph this API exported.

Exporting

export(selector?, options?): Promise<PortableGraph>

Selectors — what to export

Omit the selector to export the whole brain. Otherwise pick a node set:

Scenario Selector
Just an item (or items) { ids: ['a', 'b'] }
A collection + its children { collection: collectionId } (alias memberOf)
A connected neighbourhood { connected: { from: id, depth: 2, verbs?, direction? } }
A VFS directory / file (+ subtree) { vfsPath: '/docs', recursive?: true }
Everything matching a predicate { type, subtype, where, service }
The whole brain (omit)

The selector reuses find()'s grammar — "export what find() would match, minus ranking and limit." Structural and predicate selectors compose — a structural selector picks the nodes, and predicate keys then filter them:

// Members of a collection whose status is "open"
await data.export({ collection: collectionId, where: { status: 'open' } })

If you already have results from find(), export exactly those with the ids selector:

const hits = await brain.find({ type: NounType.Document })
await data.export({ ids: hits.map(r => r.id) })

Options — how to serialize

Option Default Effect
includeVectors false Carry embedding vectors verbatim. Off ⇒ import() re-embeds from data.
includeContent false Include VFS file bytes in blobs so files round-trip byte-identically.
includeSystem false Include system entities such as the VFS root.
edges 'induced' 'induced' (both endpoints in the set), 'incident' (also dangling edges, recorded in danglingIds), or 'none' (nodes only).
// A self-contained subgraph with vectors, ready to restore elsewhere
const subset = await data.export({ ids }, { includeVectors: true })

// A VFS subtree including file bytes
const tree = await data.export({ vfsPath: '/docs' }, { includeContent: true })

Importing

import(graph, options?): Promise<ImportResult>
const result = await brain.data().then(d => d.import(graph, { onConflict: 'merge' }))
// → { imported, merged, skipped, reembedded, blobsWritten, errors }
Option Default Effect
onConflict 'merge' 'merge' (update existing id in place — assemble many graphs), 'replace' (delete + recreate), or 'skip'.
reembed 'auto' 'auto' (use the carried vector, else re-embed from data) or 'never' (require a carried vector; record an error if absent).
remapIds Rewrite every id on the way in, e.g. to clone a template subgraph under fresh ids.

The default onConflict: 'merge' is what lets you assemble one working graph from many exported documents that share entity ids — re-importing an id merges rather than duplicates.

// Clone a subgraph under fresh ids (a copy, not a move)
const remap = new Map(graph.entities.map(e => [e.id, crypto.randomUUID()]))
await data.import(graph, { remapIds: id => remap.get(id) ?? id })

The PortableGraph format

{
  "format": "brainy-portable-graph",
  "formatVersion": 1,                 // import gates on this (cross-version migration)
  "brainyVersion": "7.32.0",
  "createdAt": "2026-06-16T…Z",
  "embedding": { "model": "all-MiniLM-L6-v2", "dimensions": 384 },
  "selector": {  },                  // echoes what was exported (provenance)
  "entities": [
    {
      "id": "…", "type": "Document", "subtype": "invoice",
      "data": "…",                    // the embedding source
      "confidence": 1, "weight": 1, "service": "…",
      "vector": [  ],                // only with includeVectors
      "metadata": {  }               // custom fields only (reserved fields are top-level)
    }
  ],
  "relations": [
    { "id":"…", "from":"…", "to":"…", "type":"Contains", "subtype":"…",
      "weight":1, "confidence":1, "metadata": {  } }
  ],
  "blobs": { "<sha256>": "<base64>" }, // only with includeContent
  "danglingIds": [ "…" ],             // only with edges:'incident'
  "stats": { "entityCount": 0, "relationCount": 0, "blobCount": 0, "vectorDimensions": 384 }
}

Standard fields (subtype, data, confidence, weight, service) sit at the top level of each entity; metadata holds only custom user fields — mirroring the in-memory Entity shape, so import() maps each field to its dedicated parameter.

Cross-version (7.x → 8.0)

Because the document is shared and versioned, a graph written by 7.x imports into 8.0: formatVersion is read forward, subtype is carried so 8.0 re-types correctly, and the same 384-dimension model on both lines means includeVectors:false re-embeds identically (or true carries vectors verbatim). The format is current-state — if you need a whole-brain snapshot with generation history, that is a separate native facility (db.persist() / Brainy.load() on 8.0).

VFS

VFS directories are Collection entities and files are entities linked by Contains, so the whole filesystem (or any subtree) exports through the vfsPath selector:

await data.export({ vfsPath: '/' },        { includeContent: true }) // all VFS + bytes
await data.export({ vfsPath: '/docs' },    { includeContent: true }) // one directory
await data.export({ vfsPath: '/a/b.txt' }, { includeContent: true }) // one file