David Snelling
373a48122d
The export()/import() document type was named BackupData, but it is not a backup: it is the portable, versioned, partial-or-whole interchange representation of a graph (entities + relations + optional vectors/blobs) — the unit Brainy exports for transport between instances, versions, and products, and the payload other artifacts embed. The actual backup is persist()/load() (the native whole-brain, generation-preserving snapshot), so "Backup*" actively collided with that concept. Rename every developer-visible symbol, file, doc, JSDoc and comment: - BackupData→PortableGraph, BackupEntity→PortableGraphEntity, BackupRelation→PortableGraphRelation, BackupReader/Writer→PortableGraphReader/Writer, BackupValidation→PortableGraphValidation, isBackupData→isPortableGraph, validateBackup→validatePortableGraph, BACKUP_FORMAT[_VERSION]→PORTABLE_GRAPH_FORMAT[_VERSION]. - src/db/backup.ts → src/db/portableGraph.ts; test → db-portable-graph.test.ts. - Guide, api/README, RELEASES updated. The on-the-wire `format` tag is renamed 'brainy-backup' → 'brainy-portable-graph': the format was introduced in 7.32.0 and has not been adopted by any consumer, so there are no stored documents to stay compatible with — a clean rename beats carrying a legacy tag. No deprecated aliases (nothing to alias). 7.x shipped the same rename as 7.32.2. 1471 unit green; build clean; db-portable-graph.test.ts 17/17.
181 lines
8.2 KiB
Markdown
181 lines
8.2 KiB
Markdown
---
|
|
title: Export & Import (portable graph)
|
|
slug: guides/export-and-import
|
|
public: true
|
|
category: guides
|
|
template: guide
|
|
order: 9
|
|
description: Export part or all of a brain to a portable, versioned PortableGraph document and import it back — by id, collection, connected neighbourhood, VFS subtree, predicate, or the whole brain. export() lives on the immutable Db, so asOf()/with() give time-travel and what-if exports.
|
|
next:
|
|
- guides/subtypes-and-facets
|
|
- api/README
|
|
---
|
|
|
|
# Export & Import (portable graph)
|
|
|
|
Brainy serializes part or all of a brain — an item, a collection, a connected
|
|
neighbourhood, a VFS subtree, a predicate match, or the whole brain — into a single
|
|
versioned JSON document (`PortableGraph`), and restores it.
|
|
|
|
```typescript
|
|
const graph = await brain.export() // whole brain → PortableGraph
|
|
await brain.import(graph) // restore (merge by id, re-embed if no vectors)
|
|
```
|
|
|
|
It is **portable** (human-readable JSON), **versioned** (`formatVersion`, so a document
|
|
written by 7.x imports cleanly into 8.0), and **current-state** (the entities and edges as
|
|
they are now — no generation history). Use it for portable artifacts, partial exports,
|
|
cross-environment moves, and version upgrades.
|
|
|
|
`export()` is a method on the **immutable `Db` value**, so it composes with every way of
|
|
obtaining one:
|
|
|
|
```typescript
|
|
brain.export(sel) // = brain.now().export(sel)
|
|
;(await brain.asOf(gen)).export(sel) // time-travel export (a past generation)
|
|
brain.now().with(ops).export(sel) // what-if export (a speculative state)
|
|
```
|
|
|
|
## When to use which
|
|
|
|
| You want… | Use |
|
|
|-----------|-----|
|
|
| A portable, partial-or-whole, cross-version graph document | **`brain.export()` / `brain.import()`** (this guide) |
|
|
| A whole-brain snapshot **with generation history** | `brain.now().persist(path)` / `Brainy.load(path)` (native) |
|
|
| To ingest a CSV / PDF / Excel / JSON **file** as new entities | `brain.import(file)` — see [Import Anything](./import-anything.md) |
|
|
|
|
`import()` is **polymorphic**: hand it a `PortableGraph` and it does the graph round-trip;
|
|
hand it a file/buffer and it does foreign-file ingestion (dispatched on the document's
|
|
`format: 'brainy-portable-graph'` tag).
|
|
|
|
## Exporting
|
|
|
|
```typescript
|
|
brain.export(selector?, options?): Promise<PortableGraph>
|
|
// (also on any Db: brain.now().export(...), (await brain.asOf(g)).export(...))
|
|
```
|
|
|
|
### Selectors — *what* to export
|
|
|
|
Omit the selector to export the whole brain. Otherwise pick a node set:
|
|
|
|
| Scenario | Selector |
|
|
|----------|----------|
|
|
| Just an item (or items) | `{ ids: ['a', 'b'] }` |
|
|
| A collection + its children | `{ collection: collectionId }` (alias `memberOf`) |
|
|
| A connected neighbourhood | `{ connected: { from: id, depth: 2, verbs?, direction? } }` |
|
|
| A VFS directory / file (+ subtree) | `{ vfsPath: '/docs', recursive?: true }` |
|
|
| Everything matching a predicate | `{ type, subtype, where, service, visibility }` |
|
|
| The whole brain | *(omit)* |
|
|
|
|
The selector reuses `find()`'s grammar — *"export what `find()` would match, minus ranking
|
|
and limit."* Structural and predicate selectors **compose**:
|
|
|
|
```typescript
|
|
// Members of a collection whose status is "open"
|
|
await brain.export({ collection: collectionId, where: { status: 'open' } })
|
|
|
|
// Already have find() results? Export exactly those with the ids selector
|
|
const hits = await brain.find({ type: NounType.Document })
|
|
await brain.export({ ids: hits.map(r => r.id) })
|
|
```
|
|
|
|
### Options — *how* to serialize
|
|
|
|
| Option | Default | Effect |
|
|
|--------|---------|--------|
|
|
| `includeVectors` | `false` | Carry embedding vectors verbatim. Off ⇒ `import()` re-embeds from `data`. |
|
|
| `includeContent` | `false` | Include VFS file bytes in `blobs` so files round-trip byte-identically. |
|
|
| `includeSystem` | `false` | Include `visibility:'system'` entities such as the VFS root. |
|
|
| `edges` | `'induced'` | `'induced'` (both endpoints in the set), `'incident'` (also dangling edges, recorded in `danglingIds`), or `'none'` (nodes only). |
|
|
|
|
## Importing
|
|
|
|
```typescript
|
|
brain.import(graph, options?): Promise<ImportResult>
|
|
```
|
|
|
|
The whole graph is applied as **one atomic transaction** — it advances the brain exactly
|
|
one generation, or none on failure.
|
|
|
|
```typescript
|
|
const result = await brain.import(graph, { onConflict: 'merge' })
|
|
// → { imported, merged, skipped, reembedded, blobsWritten, errors }
|
|
```
|
|
|
|
| Option | Default | Effect |
|
|
|--------|---------|--------|
|
|
| `onConflict` | `'merge'` | `'merge'` (update existing id in place — assemble many exported graphs), `'replace'` (delete + recreate), or `'skip'`. |
|
|
| `reembed` | `'auto'` | `'auto'` (use the carried vector, else re-embed from `data`) or `'never'` (require a carried vector; record an error if absent). |
|
|
| `remapIds` | — | Rewrite every id on the way in, e.g. to clone a template subgraph under fresh ids. |
|
|
| `meta` | — | Transaction metadata recorded in the tx-log alongside the new generation. |
|
|
|
|
The default `onConflict: 'merge'` lets you assemble one working graph from many exported
|
|
documents that share entity ids — re-importing an id merges rather than duplicates.
|
|
|
|
## The `PortableGraph` format
|
|
|
|
```jsonc
|
|
{
|
|
"format": "brainy-portable-graph", // identifies the document type
|
|
"formatVersion": 1, // import gates on this (cross-version migration)
|
|
"brainyVersion": "8.0.0",
|
|
"createdAt": "2026-06-16T…Z",
|
|
"embedding": { "model": "all-MiniLM-L6-v2", "dimensions": 384 },
|
|
"selector": { … }, // echoes what was exported (provenance)
|
|
"entities": [
|
|
{
|
|
"id": "…", "type": "Document", "subtype": "invoice", "visibility": "public",
|
|
"data": "…", // the embedding source
|
|
"confidence": 1, "weight": 1, "service": "…",
|
|
"vector": [ … ], // only with includeVectors
|
|
"metadata": { … } // custom fields only (reserved fields are top-level)
|
|
}
|
|
],
|
|
"relations": [
|
|
{ "id":"…", "from":"…", "to":"…", "type":"Contains", "subtype":"…",
|
|
"weight":1, "confidence":1, "metadata": { … } }
|
|
],
|
|
"blobs": { "<sha256>": "<base64>" }, // only with includeContent
|
|
"danglingIds": [ "…" ], // only with edges:'incident'
|
|
"stats": { "entityCount": 0, "relationCount": 0, "blobCount": 0, "vectorDimensions": 384 }
|
|
}
|
|
```
|
|
|
|
Standard fields (`subtype`, `visibility`, `data`, `confidence`, `weight`, `service`) sit at
|
|
the top level of each entity; `metadata` holds **only** custom user fields — mirroring the
|
|
in-memory `Entity` shape, so `import()` maps each field to its dedicated parameter. The
|
|
TypeScript types (`PortableGraph`, `PortableGraphEntity`, `PortableGraphRelation`, `ExportSelector`,
|
|
`ExportOptions`, `ImportOptions`, `ImportResult`) are exported from the package root.
|
|
|
|
## Generations & time-travel
|
|
|
|
The portable document is **current-state** — it never embeds generation history (that keeps
|
|
it cross-version-portable). History lives where it's queryable:
|
|
|
|
- **During a session:** `brain.asOf(g)` / `brain.now().with(ops)` on the live brain. Because
|
|
`export()` is on the `Db`, `(await brain.asOf(g)).export()` serializes a *past* generation
|
|
and `brain.now().with(ops).export()` serializes a *speculative* one.
|
|
- **A whole-brain snapshot with history:** `brain.now().persist(path)` / `Brainy.load(path)`
|
|
(native, generation-preserving) — a separate facility from this portable format.
|
|
|
|
Note: only `transact()` (and the write shortcuts that commit through it) advances a
|
|
generation, so time-travel export differs across transaction boundaries.
|
|
|
|
## Cross-version (7.x → 8.0)
|
|
|
|
Because the document is shared and versioned, a PortableGraph written by 7.x imports into 8.0:
|
|
`formatVersion` is read forward, `subtype` is carried so 8.0 re-types correctly, and the
|
|
same 384-dimension model on both lines means `includeVectors:false` re-embeds identically
|
|
(or `true` carries vectors verbatim).
|
|
|
|
## VFS
|
|
|
|
VFS directories are `Collection` entities and files are entities linked by `Contains`, so
|
|
the whole filesystem (or any subtree) exports through the `vfsPath` selector:
|
|
|
|
```typescript
|
|
await brain.export({ vfsPath: '/' }, { includeContent: true }) // all VFS + bytes
|
|
await brain.export({ vfsPath: '/docs' }, { includeContent: true }) // one directory
|
|
await brain.export({ vfsPath: '/a/b.txt' }, { includeContent: true }) // one file
|
|
```
|