2026-02-19 17:04:05 -08:00
---
title: API Reference
slug: api/reference
public: true
category: api
template: api
order: 1
description: Complete API reference for all Brainy methods — add, find, relate, update, delete, batch operations, branching, entity versioning, VFS, neural API, and more.
next:
- getting-started/quick-start
- guides/find-system
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
# 🧠 Brainy API Reference
2025-08-26 12:32:21 -07:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
> **Complete API documentation for Brainy**
2026-01-06 14:52:12 -08:00
> Zero Configuration • Triple Intelligence • Git-Style Branching • Entity Versioning • Candle WASM Embeddings
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
**Updated:** 2026-01-06
2025-11-02 10:58:52 -08:00
**All APIs verified against actual code**
---
2025-08-26 12:32:21 -07:00
## Quick Start
```typescript
2025-11-02 10:58:52 -08:00
import { Brainy, NounType, VerbType } from '@soulcraft/brainy '
2025-08-26 12:32:21 -07:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
const brain = new Brainy() // Zero config!
await brain.init() // VFS auto-initialized!
2025-08-26 12:32:21 -07:00
// Add data (text auto-embeds!)
2025-11-02 10:58:52 -08:00
const id = await brain.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'The future of AI is here',
type: NounType.Concept,
metadata: { category: 'technology' }
2025-11-02 10:58:52 -08:00
})
2025-08-26 12:32:21 -07:00
// Search with Triple Intelligence
const results = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'artificial intelligence',
where: { year: { greaterThan: 2020 } },
connected: { from: id, depth: 2 }
2025-08-26 12:32:21 -07:00
})
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Fork for safe experimentation
2025-11-02 10:58:52 -08:00
const experiment = await brain.fork('test-feature')
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
await experiment.add({ data: 'test', type: NounType.Document })
2025-11-02 10:58:52 -08:00
await experiment.commit({ message: 'Add test data' })
2025-11-04 11:19:02 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Entity versioning
2025-11-04 11:19:02 -08:00
await brain.versions.save(id, { tag: 'v1.0', description: 'Initial version' })
await brain.update(id, { category: 'AI' })
await brain.versions.save(id, { tag: 'v2.0' })
2025-08-26 12:32:21 -07:00
```
2025-11-02 10:58:52 -08:00
---
2025-08-26 12:32:21 -07:00
## Core Concepts
2025-11-02 10:58:52 -08:00
### 🧬 Entities (Nouns)
Semantic vectors with metadata and relationships - the fundamental data unit in Brainy.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
### 🔗 Relationships (Verbs)
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
Typed connections between entities with optional `data` and `metadata` - building knowledge graphs.
### 📊 Data vs Metadata
- **`data` **: Content embedded into vectors. Searchable via **semantic similarity** (HNSW) and **hybrid text+semantic** search. NOT queryable via `where` filters.
- **`metadata` **: Structured fields indexed by MetadataIndex. Queryable via `where` filters in `find()` .
See ** [Data Model ](../DATA_MODEL.md )** for the full explanation.
2025-08-26 12:32:21 -07:00
### 🧠 Triple Intelligence
Vector search + Graph traversal + Metadata filtering in one unified query.
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### 🌳 Git-Style Branching
feat: ID-first storage architecture + remove memory-unsafe APIs (v6.0.0)
BREAKING CHANGES:
**ID-First Storage Paths**
- Direct O(1) entity access without type lookups
- Before: entities/nouns/{TYPE}/metadata/{SHARD}/{ID}.json
- After: entities/nouns/{SHARD}/{ID}/metadata.json
- Migration handled automatically on first init()
**Removed Memory-Unsafe APIs**
- Removed brain.merge() - loaded all entities into memory
- Removed brain.diff() - loaded all entities into memory
- Removed brain.data().backup() - loaded all entities into memory
- Removed brain.data().restore() - depended on backup()
- Removed CLI commands: backup, restore, cow merge
**Migration Paths**
- merge() → Use checkout() or manually copy entities with pagination
- diff() → Use asOf() with manual paginated comparison
- backup() → Use fork() for instant COW snapshots
- restore() → Use checkout() to switch to snapshot branch
Core Improvements:
- ✅ All 8 storage adapters properly call super.init()
- ✅ GraphAdjacencyIndex integration in BaseStorage.init()
- ✅ Fixed ID-first path bugs (vector.json → vectors.json)
- ✅ Fixed MemoryStorage.initializeCounts() for ID-first paths
- ✅ New VFS APIs: du(), access(), find()
- ✅ Comprehensive documentation with migration guides
Storage Adapters Fixed:
- MemoryStorage, FileSystemStorage, AzureBlobStorage
- GCSStorage, R2Storage, S3CompatibleStorage
- OPFSStorage, HistoricalStorageAdapter
Files Changed: 28 files, +1,075/-1,933 lines (net -858)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 16:46:11 -08:00
Fork, experiment, and commit - Snowflake-style copy-on-write isolation.
2025-08-26 12:32:21 -07:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### 📜 Entity Versioning
2025-11-04 11:19:02 -08:00
Time-travel and history tracking for individual entities - Git-like version control with content-addressable storage.
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
## Table of Contents
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
- [Core CRUD Operations ](#core-crud-operations )
- [Search & Query ](#search--query )
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
- [Aggregation Engine ](#aggregation-engine )
2025-11-02 10:58:52 -08:00
- [Relationships ](#relationships )
- [Batch Operations ](#batch-operations )
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- [Branch Management ](#branch-management )
- [Entity Versioning ](#entity-versioning )
2025-11-02 10:58:52 -08:00
- [Virtual Filesystem (VFS) ](#virtual-filesystem-vfs )
- [Neural API ](#neural-api )
- [Import & Export ](#import--export )
- [Configuration ](#configuration )
- [Storage Adapters ](#storage-adapters )
- [Utility Methods ](#utility-methods )
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- [Embedding & Analysis APIs ](#embedding--analysis-apis )
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
- [Type System Reference ](#type-system-reference )
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
---
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
## Core CRUD Operations
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
### `add(params)` → `Promise<string>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
Add a single entity to the database.
2025-08-26 12:32:21 -07:00
```typescript
2025-11-02 10:58:52 -08:00
const id = await brain.add({
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
data: 'JavaScript is a programming language', // Text or pre-computed vector
type: NounType.Concept, // Required: Entity type
subtype: 'language', // Optional: sub-classification
metadata: { // Optional: queryable fields
category: 'programming',
year: 1995
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
}
2025-08-26 12:32:21 -07:00
})
```
2025-11-02 10:58:52 -08:00
**Parameters:**
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `data` : `string | number[]` - Content to embed (text auto-embeds) or pre-computed vector
2025-11-02 10:58:52 -08:00
- `type` : `NounType` - Entity type (required)
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
- `subtype?` : `string` - Per-product sub-classification within the NounType (top-level standard field, indexed on the fast path). See [Subtypes & Facets ](../guides/subtypes-and-facets.md ).
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `metadata?` : `object` - Structured queryable fields (indexed by MetadataIndex, used in `where` filters)
- `id?` : `string` - Custom ID (auto-generated UUID if not provided)
- `vector?` : `number[]` - Pre-computed vector (skips auto-embedding)
- `confidence?` : `number` - Type classification confidence (0-1)
- `weight?` : `number` - Entity importance/salience (0-1)
feat: per-entity _rev + update({ ifRev }) CAS + add({ ifAbsent })
Optimistic concurrency for multi-writer coordination — the read-then-CAS
lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the
SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships
cleanly without painting the 8.0 transact() API into a corner.
A. PER-ENTITY _rev FIELD
- src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`;
STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly.
- src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`;
Result<T> mirrors it on the convenience flatten layer.
- src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities
always start at 1.
- src/brainy.ts update() — reads currentRev off the persisted metadata
(falls back to 1 for pre-7.31.0 entities with no _rev), writes
`_rev: currentRev + 1` into the updated metadata. Every successful update()
bumps by exactly 1.
- src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull
_rev out of the storage metadata and surface it at the top level of Entity.
Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers
see a consistent value.
- src/storage/baseStorage.ts — six destructure sites updated to pull _rev
out of the metadata bag so it doesn't leak into customMetadata. Noun-side
returns surface _rev; verb-side destructures correctly but doesn't expose
it on HNSWVerbWithMetadata (verb CAS is future work).
- src/brainy.ts createResult() — flattens entity._rev onto the Result for
backward compat with the existing convenience-field layer.
B. update({ ifRev }) OPTIMISTIC CONCURRENCY
- src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`.
- src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected,
actual }`. Message names the recipe: refetch with brain.get() and retry
with the latest _rev. Subclass of Error.
- src/brainy.ts update() — when params.ifRev is provided, compares against
currentRev (the rev we just read) and throws RevisionConflictError on
mismatch before any storage write. Omitting ifRev keeps the prior
unconditional-update behavior; existing consumers see no change.
- src/transaction/index.ts — exports RevisionConflictError.
- src/index.ts — public export of RevisionConflictError.
C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT
- src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`;
AddManyParams<T> mirrors it as a batch-level flag.
- src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads
storage.getNounMetadata(id); if present, returns the existing id without
writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID
can never collide).
- src/brainy.ts addMany() — propagates the batch-level ifAbsent to each
item's add() call. Per-item ifAbsent takes precedence so callers can
override individual rows.
WHAT'S NOT SHIPPED (and why)
A public brain.transaction(fn) wrapper was on the table but was cut. The
internal TransactionManager exposes raw Operation classes (SaveNounMetadata,
etc.) that take StorageAdapter as a constructor argument. A clean high-level
facade in 7.31.0 would have meant either:
(a) Delegate to brain.add() / update() / relate(). Each of those opens its
own internal transaction and commits before the closure returns. Looks
atomic, isn't — a footgun.
(b) Thread an optional `tx?` through every internal write site in
brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of
real refactor with regression surface, and the API shape changes again
in 8.0 anyway.
The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the
read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write
atomicity is the 8.0 brain.transact() use case where the Datomic-style
immutable Db makes it atomic by construction — that's the right home.
TESTS
- New tests/integration/rev-and-ifabsent.test.ts (18 tests):
- _rev initialization (1 on add) and surface on get fast/full paths + find
- _rev auto-bump on update across multiple writes
- update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev
- add({ ifAbsent }) writes when absent / no-op when present / id-required
- addMany({ ifAbsent }) propagation + per-item override
- SDK-scheduler scenario: two concurrent CAS updates, one wins one throws
- Unit suite unchanged: 1468/1468.
- All integration subtype + verb + strict + find-limits + new rev suites
pass: 96/96.
DOCS
- New docs/guides/optimistic-concurrency.md (public: true) — full reference:
the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev
interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0.
- docs/api/README.md — add() and update() entries get the new params + tips
pointing at the new guide.
- RELEASES.md v7.31.0 entry.
CORTEX COMPATIBILITY
Zero changes required. _rev is a metadata column already supported by
NativeColumnStore. The auto-bump runs in Brainy JS before any storage call;
Cortex never sees the per-entity counter.
8.0 FORWARD-COMPAT
_rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign
unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS
on top of the same per-entity mechanism — per-entity for single-record
patterns (job locks, idempotent state machines), generation-based for
"did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All integration suites including new rev-and-ifabsent: 96/96
- npm run build: clean
- Closed-source product reference audit: clean
2026-06-09 10:04:24 -07:00
- `ifAbsent?` : `boolean` - By-ID idempotent insert. When `true` AND a custom `id` is supplied AND an entity with that `id` already exists, returns the existing `id` without writing (no throw, no overwrite). Ignored without `id` . See [guides/optimistic-concurrency ](../guides/optimistic-concurrency.md ).
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
> **`data`** is embedded into vectors for semantic search. **`metadata`** is indexed for `where` filters. See [Data Model](../DATA_MODEL.md).
2025-11-02 10:58:52 -08:00
fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors
Brainy 7.30 shipped opt-in subtype enforcement; SDK 3.20.0 then registered
SDK_CORE_VOCABULARY on every consumer's brain (Event, Collection, Message,
Contract, Media, Document NounTypes). On 2026-06-08 Venue's /book flow went 500
because their brain.add({ type: NounType.Event, ... }) call sites lacked
subtype. An audit of Brainy's OWN source revealed 14 HIGH-risk internal write
paths that also omit subtype — any consumer running the same vocabulary would
have hit Brainy's infrastructure paths next. 7.30.1 closes both gaps before
8.0 makes strict mode the default.
Additive across the board. Zero behavior change for consumers not using strict
mode. Every change is JS-side — Cortex needs no work for 7.30.1.
NEW — brain.audit() diagnostic
- Read-only method walking storage.getNouns() / getVerbs() pagination
- Returns { entitiesWithoutSubtype: { type: count }, relationshipsWithoutSubtype,
total, scanned, recommendation }
- VFS infrastructure entities excluded by default (they bypass enforcement via
isVFSEntity marker); pass { includeVFS: true } to surface them
- The companion to migrateField (7.x) and fillSubtypes (8.0): tells consumers
exactly what would break under strict enforcement, deterministically
NEW — Improved enforcement error messages
- Caller's source location extracted from Error().stack so users see their own
call site, not a Brainy internal frame
- Specific guidance branches: registered vocabulary → "Pass one of: a, b, c";
brain-wide strict mode → mentions the except clause; otherwise → registration
recipe via brain.requireSubtype()
- Documentation link to the canonical migration recipe
- Same shape for noun and verb enforcement
NEW — CLI --subtype flag
- brainy add and brainy relate gain -s/--subtype <value>
- Defaults to 'cli-add' / 'cli-relate' so the CLI works against strict-mode
brains without the user needing to know the vocabulary in advance
INTERNAL — every Brainy write path now sets subtype
- VFS Contains edges (5 sites at lines 503/905/1694/1772/1886) → 'vfs-contains'
- VFS symlink entity → 'vfs-symlink' (NEW — distinct from 'vfs-file')
- VFS copy-file → preserves source subtype, falls back to 'vfs-file'
- VFS symlink also adopts the isVFSEntity infrastructure marker so it bypasses
enforcement in strict mode
- Aggregation materializer (Measurement entities) → 'materialized-aggregate'
- ImportCoordinator (3 sites): document → 'import-source'; entities →
options.defaultSubtype ?? 'imported'; placeholder → 'import-placeholder'
- SmartImportOrchestrator (4 entity sites + 2 batch relate sites): same
precedence (extractor → options.defaultSubtype → 'imported')
- EntityDeduplicator → candidate.subtype ?? 'imported'
- UniversalImportAPI → extractor → 'extracted' for both entities and relations
- NeuralImport → adds defaultSubtype to NeuralImportOptions; precedence same
- GoogleSheetsIntegration → request body 'subtype' ?? 'imported-from-sheets'
- ODataIntegration → request body 'Subtype' ?? 'imported-from-odata'
- MCP client message storage → 'mcp-message' (also fixes pre-existing missing
data field and missing type by aliasing from the prior text field)
Side-effect fix: storage.getNouns() paginated now surfaces subtype to top-level
- Single-noun getNoun() already did this in 7.30; the paginated path was missed
- Without this fix brain.audit() saw missing subtype on entities that actually
had one (caught by the strict-mode self-test before release)
NEW — tests/integration/strict-mode-self-test.test.ts (13 tests)
- Creates a brain under the exact SDK_CORE_VOCABULARY shape Venue hit + brain-
wide strict mode
- Exercises every internal Brainy path: VFS root + mkdir + writeFile + cp + mv
+ ln + symlink; aggregation engine; audit diagnostic with includeVFS toggle
- Validates error message UX: caller location, vocabulary guidance, brain-wide
strict mode guidance, off-vocabulary value reporting
Docs
- New "Strict mode in practice" section in docs/guides/subtypes-and-facets.md
covering the SDK_CORE_VOCABULARY pattern, 4-step migration recipe
(audit → migrateField → hand-fix → re-audit), the Brainy-internal label
reference table, and an 8.0 forward-look on fillSubtypes()
- docs/api/README.md: new audit() entry, strict-mode tips on add() and relate()
- RELEASES.md: full 7.30.1 entry
Cortex parity (forward-looking, not blocking 7.30.1)
- 6th open question added to .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md: native
fast path for audit() and fillSubtypes() via column-store null-subtype
bitmap for billion-scale brains
- Cortex should add a parity test mirroring strict-mode-self-test.test.ts
against their native paths to catch any latent bug where native writes
bypass JS validation
- Brainy-internal subtype labels become a documented part of the 8.0 contract
(useful for Cortex telemetry surfacing Brainy-managed infrastructure %)
Verification
- npx tsc --noEmit: clean
- npm test: 1468/1468 unit
- 7.29 noun integration suite: 26/26 (no regression)
- 7.30 verb subtype + enforcement integration suite: 30/30 (no regression)
- New strict-mode-self-test integration suite: 13/13
- npm run build: clean
- Closed-source product reference audit: clean
Addresses VE-SUBTYPE-MIGRATION (Venue's reported request) and ships internal
labels Venue did NOT ask for but that would have broken them next under their
own vocabulary registration.
2026-06-08 11:31:47 -07:00
> **Strict-mode tip:** if a vocabulary is registered for your `type` (via `brain.requireSubtype()` or by an SDK that wraps Brainy), you must pass a matching `subtype`. Run `await brain.audit()` to inventory pre-existing gaps before enabling strict mode; see the [migration recipe](../guides/subtypes-and-facets.md#strict-mode-in-practice-for-sdk-style-vocabulary-consumers).
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<string>` - Entity ID
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
### `get(id)` → `Promise<Entity | null>`
Retrieve a single entity by ID.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
```typescript
const entity = await brain.get(id)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(entity?.data) // Original data
console.log(entity?.metadata) // Metadata
console.log(entity?.vector) // Embedding vector
2025-11-02 10:58:52 -08:00
```
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Parameters:**
- `id` : `string` - Entity ID
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<Entity | null>` - Entity or null if not found
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
### `update(params)` → `Promise<void>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
Update an existing entity.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
```typescript
await brain.update({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
id: entityId,
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
data: 'Updated content', // Optional: new data
subtype: 'archived', // Optional: change sub-classification
metadata: { updated: true } // Optional: new metadata (merges)
2025-11-02 10:58:52 -08:00
})
```
**Parameters:**
- `id` : `string` - Entity ID
- `data?` : `string | number[]` - New data/vector
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
- `type?` : `NounType` - Change entity type
- `subtype?` : `string` - Change subtype (omit to preserve existing)
- `metadata?` : `object` - Metadata to merge (or replace with `merge: false` )
- `confidence?` : `number` - Update classification confidence
- `weight?` : `number` - Update entity importance
feat: per-entity _rev + update({ ifRev }) CAS + add({ ifAbsent })
Optimistic concurrency for multi-writer coordination — the read-then-CAS
lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the
SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships
cleanly without painting the 8.0 transact() API into a corner.
A. PER-ENTITY _rev FIELD
- src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`;
STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly.
- src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`;
Result<T> mirrors it on the convenience flatten layer.
- src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities
always start at 1.
- src/brainy.ts update() — reads currentRev off the persisted metadata
(falls back to 1 for pre-7.31.0 entities with no _rev), writes
`_rev: currentRev + 1` into the updated metadata. Every successful update()
bumps by exactly 1.
- src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull
_rev out of the storage metadata and surface it at the top level of Entity.
Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers
see a consistent value.
- src/storage/baseStorage.ts — six destructure sites updated to pull _rev
out of the metadata bag so it doesn't leak into customMetadata. Noun-side
returns surface _rev; verb-side destructures correctly but doesn't expose
it on HNSWVerbWithMetadata (verb CAS is future work).
- src/brainy.ts createResult() — flattens entity._rev onto the Result for
backward compat with the existing convenience-field layer.
B. update({ ifRev }) OPTIMISTIC CONCURRENCY
- src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`.
- src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected,
actual }`. Message names the recipe: refetch with brain.get() and retry
with the latest _rev. Subclass of Error.
- src/brainy.ts update() — when params.ifRev is provided, compares against
currentRev (the rev we just read) and throws RevisionConflictError on
mismatch before any storage write. Omitting ifRev keeps the prior
unconditional-update behavior; existing consumers see no change.
- src/transaction/index.ts — exports RevisionConflictError.
- src/index.ts — public export of RevisionConflictError.
C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT
- src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`;
AddManyParams<T> mirrors it as a batch-level flag.
- src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads
storage.getNounMetadata(id); if present, returns the existing id without
writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID
can never collide).
- src/brainy.ts addMany() — propagates the batch-level ifAbsent to each
item's add() call. Per-item ifAbsent takes precedence so callers can
override individual rows.
WHAT'S NOT SHIPPED (and why)
A public brain.transaction(fn) wrapper was on the table but was cut. The
internal TransactionManager exposes raw Operation classes (SaveNounMetadata,
etc.) that take StorageAdapter as a constructor argument. A clean high-level
facade in 7.31.0 would have meant either:
(a) Delegate to brain.add() / update() / relate(). Each of those opens its
own internal transaction and commits before the closure returns. Looks
atomic, isn't — a footgun.
(b) Thread an optional `tx?` through every internal write site in
brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of
real refactor with regression surface, and the API shape changes again
in 8.0 anyway.
The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the
read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write
atomicity is the 8.0 brain.transact() use case where the Datomic-style
immutable Db makes it atomic by construction — that's the right home.
TESTS
- New tests/integration/rev-and-ifabsent.test.ts (18 tests):
- _rev initialization (1 on add) and surface on get fast/full paths + find
- _rev auto-bump on update across multiple writes
- update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev
- add({ ifAbsent }) writes when absent / no-op when present / id-required
- addMany({ ifAbsent }) propagation + per-item override
- SDK-scheduler scenario: two concurrent CAS updates, one wins one throws
- Unit suite unchanged: 1468/1468.
- All integration subtype + verb + strict + find-limits + new rev suites
pass: 96/96.
DOCS
- New docs/guides/optimistic-concurrency.md (public: true) — full reference:
the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev
interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0.
- docs/api/README.md — add() and update() entries get the new params + tips
pointing at the new guide.
- RELEASES.md v7.31.0 entry.
CORTEX COMPATIBILITY
Zero changes required. _rev is a metadata column already supported by
NativeColumnStore. The auto-bump runs in Brainy JS before any storage call;
Cortex never sees the per-entity counter.
8.0 FORWARD-COMPAT
_rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign
unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS
on top of the same per-entity mechanism — per-entity for single-record
patterns (job locks, idempotent state machines), generation-based for
"did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All integration suites including new rev-and-ifabsent: 96/96
- npm run build: clean
- Closed-source product reference audit: clean
2026-06-09 10:04:24 -07:00
- `ifRev?` : `number` - Optimistic-concurrency check. When provided, the update throws `RevisionConflictError` if the persisted entity's `_rev` no longer equals `ifRev` . See [guides/optimistic-concurrency ](../guides/optimistic-concurrency.md ).
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<void>`
2025-08-26 12:32:21 -07:00
feat: per-entity _rev + update({ ifRev }) CAS + add({ ifAbsent })
Optimistic concurrency for multi-writer coordination — the read-then-CAS
lock pattern + idempotent-bootstrap singleton inserts. Lands the surface the
SDK scheduler asked for in BRAINY-EXPOSE-TRANSACTIONS, scoped to what ships
cleanly without painting the 8.0 transact() API into a corner.
A. PER-ENTITY _rev FIELD
- src/coreTypes.ts — HNSWNounWithMetadata gains optional `_rev?: number`;
STANDARD_ENTITY_FIELDS includes it so resolveEntityField routes correctly.
- src/types/brainy.types.ts — Entity<T> gains optional `_rev?: number`;
Result<T> mirrors it on the convenience flatten layer.
- src/brainy.ts add() — initializes `_rev: 1` in storageMetadata. New entities
always start at 1.
- src/brainy.ts update() — reads currentRev off the persisted metadata
(falls back to 1 for pre-7.31.0 entities with no _rev), writes
`_rev: currentRev + 1` into the updated metadata. Every successful update()
bumps by exactly 1.
- src/brainy.ts convertNounToEntity + convertMetadataToEntity — both pull
_rev out of the storage metadata and surface it at the top level of Entity.
Pre-7.31.0 entities (no _rev in storage) read as `_rev: 1` so consumers
see a consistent value.
- src/storage/baseStorage.ts — six destructure sites updated to pull _rev
out of the metadata bag so it doesn't leak into customMetadata. Noun-side
returns surface _rev; verb-side destructures correctly but doesn't expose
it on HNSWVerbWithMetadata (verb CAS is future work).
- src/brainy.ts createResult() — flattens entity._rev onto the Result for
backward compat with the existing convenience-field layer.
B. update({ ifRev }) OPTIMISTIC CONCURRENCY
- src/types/brainy.types.ts — UpdateParams<T> gains optional `ifRev?: number`.
- src/transaction/RevisionConflictError.ts (NEW) — carries `{ id, expected,
actual }`. Message names the recipe: refetch with brain.get() and retry
with the latest _rev. Subclass of Error.
- src/brainy.ts update() — when params.ifRev is provided, compares against
currentRev (the rev we just read) and throws RevisionConflictError on
mismatch before any storage write. Omitting ifRev keeps the prior
unconditional-update behavior; existing consumers see no change.
- src/transaction/index.ts — exports RevisionConflictError.
- src/index.ts — public export of RevisionConflictError.
C. add({ ifAbsent }) BY-ID IDEMPOTENT INSERT
- src/types/brainy.types.ts — AddParams<T> gains optional `ifAbsent?: boolean`;
AddManyParams<T> mirrors it as a batch-level flag.
- src/brainy.ts add() — when params.id AND params.ifAbsent, pre-reads
storage.getNounMetadata(id); if present, returns the existing id without
writing. No throw, no overwrite. Ignored when id is omitted (a fresh UUID
can never collide).
- src/brainy.ts addMany() — propagates the batch-level ifAbsent to each
item's add() call. Per-item ifAbsent takes precedence so callers can
override individual rows.
WHAT'S NOT SHIPPED (and why)
A public brain.transaction(fn) wrapper was on the table but was cut. The
internal TransactionManager exposes raw Operation classes (SaveNounMetadata,
etc.) that take StorageAdapter as a constructor argument. A clean high-level
facade in 7.31.0 would have meant either:
(a) Delegate to brain.add() / update() / relate(). Each of those opens its
own internal transaction and commits before the closure returns. Looks
atomic, isn't — a footgun.
(b) Thread an optional `tx?` through every internal write site in
brainy.ts (8 transactionManager.executeTransaction sites). ~2 days of
real refactor with regression surface, and the API shape changes again
in 8.0 anyway.
The SDK scheduler's actual ask (BRAINY-EXPOSE-TRANSACTIONS) is the
read-then-CAS lock pattern. _rev + ifRev solves it completely. Multi-write
atomicity is the 8.0 brain.transact() use case where the Datomic-style
immutable Db makes it atomic by construction — that's the right home.
TESTS
- New tests/integration/rev-and-ifabsent.test.ts (18 tests):
- _rev initialization (1 on add) and surface on get fast/full paths + find
- _rev auto-bump on update across multiple writes
- update({ ifRev }) pass / fail / message format / omitted / legacy-no-rev
- add({ ifAbsent }) writes when absent / no-op when present / id-required
- addMany({ ifAbsent }) propagation + per-item override
- SDK-scheduler scenario: two concurrent CAS updates, one wins one throws
- Unit suite unchanged: 1468/1468.
- All integration subtype + verb + strict + find-limits + new rev suites
pass: 96/96.
DOCS
- New docs/guides/optimistic-concurrency.md (public: true) — full reference:
the lock pattern, read-modify-write retry, idempotent bootstrap, how _rev
interacts with brain.versions, branches/fork, and VFS, what's coming in 8.0.
- docs/api/README.md — add() and update() entries get the new params + tips
pointing at the new guide.
- RELEASES.md v7.31.0 entry.
CORTEX COMPATIBILITY
Zero changes required. _rev is a metadata column already supported by
NativeColumnStore. The auto-bump runs in Brainy JS before any storage call;
Cortex never sees the per-entity counter.
8.0 FORWARD-COMPAT
_rev, ifRev, RevisionConflictError, and ifAbsent survive the 8.0 Db redesign
unchanged. 8.0 layers brain.transact(tx, { ifAtGeneration }) for whole-tx CAS
on top of the same per-entity mechanism — per-entity for single-record
patterns (job locks, idempotent state machines), generation-based for
"did the world move under me." Locked in .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md § C-6.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All integration suites including new rev-and-ifabsent: 96/96
- npm run build: clean
- Closed-source product reference audit: clean
2026-06-09 10:04:24 -07:00
> **Tip — read-then-CAS.** Every entity returned by `get()` / `find()` / `search()` carries `entity._rev` (a monotonic counter Brainy auto-bumps on every successful `update()`). Pass it back as `ifRev` to make multi-writer coordination safe without an external lock service. Full guide: [guides/optimistic-concurrency](../guides/optimistic-concurrency.md).
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
### `delete(id)` → `Promise<void>`
Delete a single entity.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
```typescript
await brain.delete(id)
```
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Parameters:**
- `id` : `string` - Entity ID
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<void>`
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
## Search & Query
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
### `find(query)` → `Promise<Result[]>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Triple Intelligence** - Vector + Graph + Metadata in ONE query.
2025-08-26 12:32:21 -07:00
```typescript
2025-11-02 10:58:52 -08:00
// Simple text search
const results = await brain.find('machine learning')
// Advanced Triple Intelligence query
const results = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'artificial intelligence', // Vector similarity
where: { // Metadata filtering
year: { greaterThan: 2020 },
category: { oneOf: ['AI', 'ML'] }
},
connected: { // Graph traversal
to: conceptId,
depth: 2,
type: VerbType.RelatedTo
},
limit: 10
2025-11-02 10:58:52 -08:00
})
2025-08-26 12:32:21 -07:00
```
2025-11-02 10:58:52 -08:00
**Parameters:**
- `query` : `string | FindParams`
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- **Simple:** Just text for vector search
- **Advanced:** Object with vector + graph + metadata filters
2025-11-02 10:58:52 -08:00
**FindParams:**
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `query?` : `string` - Text for semantic + hybrid search (searches `data` via HNSW + text index)
- `type?` : `NounType | NounType[]` - Filter by entity type(s). Alias for `where.noun` .
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
- `subtype?` : `string | string[]` - Filter by sub-classification (top-level standard field, fast path). Single string for equality, array for set membership.
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `where?` : `object` - Metadata filters. See ** [Query Operators ](../QUERY_OPERATORS.md )** for all operators.
2025-11-02 10:58:52 -08:00
- `connected?` : `object` - Graph traversal options
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `to?` : `string` - Target entity ID
- `from?` : `string` - Source entity ID
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `via?` : `VerbType | VerbType[]` - Relationship type(s) to traverse
- `type?` : `VerbType | VerbType[]` - Alias for `via`
- `depth?` : `number` - Traversal depth (default: 1)
- `direction?` : `'in' | 'out' | 'both'` - Traversal direction (default: 'both')
2025-11-02 10:58:52 -08:00
- `limit?` : `number` - Max results (default: 10)
- `offset?` : `number` - Skip results
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `orderBy?` : `string` - Field to sort by (e.g., 'createdAt', 'metadata.priority')
- `order?` : `'asc' | 'desc'` - Sort direction (default: 'asc')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `searchMode?` : `'auto' | 'text' | 'semantic' | 'hybrid'` - Search strategy:
- `'auto'` (default): Zero-config hybrid combining text + semantic search
- `'text'` : Pure keyword/text matching
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `'semantic'` /`'vector'` : Pure vector similarity
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `'hybrid'` : Explicit hybrid mode
2026-01-26 17:16:18 -08:00
- `hybridAlpha?` : `number` - Balance between text (0.0) and semantic (1.0) search. Auto-detected by query length if not specified.
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `excludeVFS?` : `boolean` - Exclude VFS entities from results (default: false)
2025-11-02 10:58:52 -08:00
fix: recalibrate find({ limit }) cap + two-tier enforcement + caller location
Brainy 7.30.0 introduced a memory-derived synchronous cap on `find({ limit })`
to prevent OOM. The cap was sound in intent but ~4x too conservative in
calibration: assumed 100 KB per result while typical entity footprint is 7-10 KB
(384-dim float32 vector ≈ 1.5 KB + standard fields + metadata). On a 900 MB
free-memory box the cap derived to 9000 — breaking common safety-cap patterns
like `find({ type, where, limit: 10_000 })` that typically return 10-500
entities. Surfaced as a runtime regression with cascading 500s degrading
production dashboards.
Three concurrent fixes:
A. RECALIBRATE THE FORMULA
- src/utils/paramValidation.ts:175,196,212 — the three memory-derived priorities
(reservedQueryMemory / containerMemory / freeMemory) all divided by
100 * 1024 * 1024 (100 KB per result, ~10-15x over conservative). Replaced
with a new MAX_LIMIT_KB_PER_RESULT = 25 constant that matches observed
entity size.
- Result: 4 GB container cap goes 10_000 → 40_000; 2 GB cap goes 5_000 →
20_000; 900 MB free-memory cap goes 9_000 → ~36_000. 100k hard ceiling
unchanged. `maxQueryLimit` / `reservedQueryMemory` constructor overrides
unchanged in behavior.
B. TWO-TIER ENFORCEMENT (warn-then-throw)
- Below cap (limit <= maxLimit): silent pass, unchanged.
- Soft tier (maxLimit < limit <= 2 * maxLimit): NEW — one-time warning per
call site (dedup keyed on caller stack frame + limit value), query
proceeds. Pre-7.30.2 code that relied on the cap silently allowing typical
safety-cap limits keeps working; the warning teaches the recipe so consumers
can fix it intentionally.
- Hard tier (limit > 2 * maxLimit): throw with the same teaching message
format. Real OOM territory; the cap stops being a recommendation and becomes
a guardrail.
- The 2x soft margin absorbs typical safety-cap patterns (limit: 10_000
against a 9 K-cap box) without disabling OOM protection. Real OOM territory
on a JS in-memory brain is hundreds of thousands of results, not 10x the
safety cap.
C. IMPROVED ERROR / WARNING MESSAGE
- Same shape as the 7.30.1 enforcement-error messages: state the problem,
name the three escape valves (maxQueryLimit / reservedQueryMemory /
pagination), include caller location, link to docs.
- Extracted findCallerLocation() helper from brainy.ts to a new
src/utils/callerLocation.ts so both the subtype enforcement (7.30.1) and
the limit enforcement (7.30.2) share one implementation without circular
imports.
DOCS
- New docs/guides/find-limits.md (public: true) — full reference: why the cap
exists, the four memory sources the auto-config considers, the three escape
valves with when-to-use-which guidance, and an explicit "pagination is the
future-proof pattern" callout (8.0 may tighten the cap further; pagination
keeps working unchanged).
- docs/api/README.md find() entry gets a one-paragraph `limit` tip + pointer
to the new guide.
- RELEASES.md v7.30.2 entry.
TESTS
- New tests/integration/find-limits.test.ts (9 tests): below-cap silent pass;
soft-tier warns once per call site (dedup verified by exercising same vs.
different source lines via wrapper closures); soft-tier message format
(names all three escape valves + docs link); soft-tier message includes
caller location; hard-tier throws; hard-tier message format same as
soft-tier; consumer maxQueryLimit override raises the cap and shifts both
tiers accordingly; pre-7.30.2 regression scenario explicitly covered.
- tests/unit/utils/memoryLimits.test.ts — 4 tests updated for the recalibrated
cap values (hardcoded expected numbers bumped 4x to match new 25 KB/result
assumption).
- tests/unit/utils/paramValidation.test.ts — auto-limit test extended to cover
the three-tier semantics (below-cap pass / soft-tier silent / hard-tier
throw).
- Existing suites unchanged: subtype-and-facets 26/26, verb-subtype-and-
enforcement 30/30, strict-mode-self-test 13/13. Unit 1468/1468.
CORTEX COMPATIBILITY
- Zero Cortex changes required. Every change is JS-side: formula recalibration
runs in ValidationConfig.constructor(), two-tier enforcement runs in
validateFindParams(), both fire before any storage / index / Cortex call.
- The new guide notes that Brainy 8.0's Datomic-style Db.find() may tighten
per-call limits to keep snapshot semantics cheap; pagination remains the
pattern that's guaranteed to keep working.
REPO-WIDE CLEANUP
Brainy is the only Soulcraft project that is open source. This commit also
scrubs closed-source product names and product-specific class/field references
from every tracked file in the repo (src/, docs/, tests/, RELEASES.md,
CHANGELOG.md). Consumer-reported bugs, regression scenarios, and release
notes now refer to "a consumer", "a downstream application", "a production
deployment", or "an internal report" — never to the named product. Two
product-named test files renamed to neutral diagnostic names. CLAUDE.md gains
a project-level guard rule documenting the policy and an example list of the
identifiers that may not appear in tracked code.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All four integration subtype + verb + strict + find-limits suites: 78/78
- npm run build: clean
- Closed-source product reference audit: clean
2026-06-08 12:34:05 -07:00
> **`limit` tip:** Brainy caps `limit` against an auto-configured maximum (based on container/free memory, ~25 KB per result). Above the cap you get a one-time warning per call site; above 2× the cap it throws. To raise the cap, pass `new Brainy({ maxQueryLimit: N })` or `{ reservedQueryMemory: bytes }`. For queries that need ALL matches, paginate with `{ limit, offset }` — that's the only pattern guaranteed to keep working across Brainy versions. See [Query Limits & Pagination](../guides/find-limits.md).
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<Result[]>` - Matching entities with scores
2025-08-26 12:32:21 -07:00
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### Hybrid Search
2026-01-26 17:16:18 -08:00
Brainy automatically combines text (keyword) and semantic (vector) search for optimal results. No configuration needed.
```typescript
// Zero-config hybrid search (just works)
const results = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'David Smith' // Finds both exact text matches AND semantically similar
2026-01-26 17:16:18 -08:00
})
// Force text-only search (exact keyword matching)
const textResults = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'exact keyword',
searchMode: 'text'
2026-01-26 17:16:18 -08:00
})
// Force semantic-only search (vector similarity)
const semanticResults = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'artificial intelligence concepts',
searchMode: 'semantic'
2026-01-26 17:16:18 -08:00
})
// Custom hybrid weighting (0 = text only, 1 = semantic only)
const customResults = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'David Smith',
hybridAlpha: 0.3 // Favor text matching
2026-01-26 17:16:18 -08:00
})
```
**How it works:**
- Short queries (1-2 words) automatically favor text matching
- Long queries (5+ words) automatically favor semantic search
- Results are combined using Reciprocal Rank Fusion (RRF)
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### Match Visibility
2026-01-26 17:16:18 -08:00
Search results include detailed match information:
```typescript
const results = await brain.find({ query: 'david the warrior' })
// Each result now includes:
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
results[0].textMatches // ["david", "warrior"] - exact query words found
results[0].textScore // 0.25 - text match quality (0-1)
results[0].semanticScore // 0.87 - semantic similarity (0-1)
results[0].matchSource // 'both' | 'text' | 'semantic'
2026-01-26 17:16:18 -08:00
```
**Use cases:**
- Highlight exact matches in UI (textMatches)
- Explain why a result ranked high (matchSource)
- Debug search behavior (separate scores)
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `highlight(params)` → `Promise<Highlight[]>` ✨
2026-01-26 17:16:18 -08:00
Zero-config highlighting for both exact matches AND semantic concepts.
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
Handles plain text, rich-text JSON (TipTap, Slate, Lexical, Draft.js, Quill), HTML, and Markdown automatically.
2026-01-26 17:16:18 -08:00
```typescript
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
// Plain text (works as before)
2026-01-26 17:16:18 -08:00
const highlights = await brain.highlight({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: "david the warrior",
text: "David Smith is a brave fighter who battles dragons"
2026-01-26 17:16:18 -08:00
})
// [
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// { text: "David", score: 1.0, position: [0, 5], matchType: 'text' },
// { text: "fighter", score: 0.78, position: [25, 32], matchType: 'semantic' },
// { text: "battles", score: 0.72, position: [37, 44], matchType: 'semantic' }
2026-01-26 17:16:18 -08:00
// ]
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Rich-text JSON (auto-detected)
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
const highlights = await brain.highlight({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: "david the warrior",
text: JSON.stringify(tiptapDocument) // TipTap, Slate, Lexical, Draft.js, Quill
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
})
// Extracts text from nodes, annotates with contentCategory:
// [
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// { text: "David", score: 1.0, matchType: 'text', contentCategory: 'title' },
// { text: "fighter", score: 0.78, matchType: 'semantic', contentCategory: 'content' }
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
// ]
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// HTML input (auto-detected)
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
const highlights = await brain.highlight({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: "warrior",
text: "< h1 > David the Warrior< / h1 > < p > A brave fighter.< / p > "
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
})
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Custom extractor for proprietary formats
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
const highlights = await brain.highlight({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: "function",
text: sourceCode,
contentExtractor: (text) => treeSitterParse(text) // Your custom parser
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
})
2026-01-26 17:16:18 -08:00
```
**Parameters:**
- `query` : `string` - The search query
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
- `text` : `string` - Text to highlight (plain text, JSON, HTML, or Markdown)
2026-01-26 17:16:18 -08:00
- `granularity?` : `'word' | 'phrase' | 'sentence'` - Highlight unit (default: 'word')
- `threshold?` : `number` - Min similarity for semantic matches (default: 0.5)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `contentType?` : `ContentType` - Optional hint: `'plaintext' | 'richtext-json' | 'html' | 'markdown'` . Skips auto-detection when provided.
- `contentExtractor?` : `(text: string) => ExtractedSegment[]` - Custom parser. Bypasses built-in detection entirely.
2026-01-26 17:16:18 -08:00
**Returns:** `Promise<Highlight[]>`
- `text` - The matched text
- `score` - Match score (1.0 for text matches, varies for semantic)
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
- `position` - [start, end] indices in extracted text
2026-01-26 17:16:18 -08:00
- `matchType` - `'text'` (exact) or `'semantic'` (concept)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `contentCategory?` - `'title' | 'annotation' | 'content' | 'value' | 'code' | 'structural'` — Role of the source text. Built-in extractors produce `'title'` , `'content'` , `'code'` . All 6 categories are available for custom parsers.
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
**Supported Rich-Text Formats:**
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
| Format | Detection | Text nodes |
|--------|-----------|------------|
| TipTap / ProseMirror | `{ type: 'doc', content: [...] }` | `{ type: 'text', text }` |
| Slate.js | `[{ type, children }]` | `{ text }` |
| Lexical | `{ root: { children } }` | `{ type: 'text', text }` |
| Draft.js | `{ blocks: [{ text }] }` | `{ text }` in block |
| Quill Delta | `{ ops: [{ insert }] }` | `{ insert }` |
| HTML | Tags like `<h1>` , `<p>` , `<code>` | Visible text content |
| Markdown | `#` headings, ` ` `` ` code blocks | Stripped markup |
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
**Timeout Protection:**
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
Semantic matching has a 10-second timeout. If embedding takes too long (e.g., WASM stall), `highlight()` returns text-only matches instead of hanging.
2026-01-26 17:16:18 -08:00
**UI Pattern:**
```typescript
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
// Style differently based on match type and content category
2026-01-26 17:16:18 -08:00
highlights.forEach(h => {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
const style = h.matchType === 'text' ? 'font-weight: bold' : 'background: yellow'
if (h.contentCategory === 'title') { /* render as heading highlight */ }
if (h.contentCategory === 'code') { /* render with code styling */ }
if (h.contentCategory === 'annotation') { /* render as comment/caption */ }
// Apply style from h.position[0] to h.position[1]
2026-01-26 17:16:18 -08:00
})
```
---
2025-11-02 10:58:52 -08:00
### Query Operators
2025-08-26 12:32:21 -07:00
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
Brainy uses clean, readable operators (BFO — Brainy Field Operators):
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
| Operator | Description | Example |
|----------|-------------|---------|
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
| `equals` / `eq` | Exact match | `{age: {equals: 25}}` |
| `notEquals` / `ne` | Not equal | `{status: {notEquals: 'deleted'}}` |
| `greaterThan` / `gt` | Greater than | `{age: {greaterThan: 18}}` |
| `greaterEqual` / `gte` | Greater or equal | `{score: {greaterEqual: 90}}` |
| `lessThan` / `lt` | Less than | `{price: {lessThan: 100}}` |
| `lessEqual` / `lte` | Less or equal | `{rating: {lessEqual: 3}}` |
| `between` | Inclusive range | `{year: {between: [2020, 2025]}}` |
| `oneOf` / `in` | In array | `{color: {oneOf: ['red', 'blue']}}` |
| `noneOf` | Not in array | `{status: {noneOf: ['deleted']}}` |
| `contains` | Array contains value | `{tags: {contains: 'ai'}}` |
| `exists` / `missing` | Field existence | `{email: {exists: true}}` |
2025-08-26 12:32:21 -07:00
| `startsWith` | String prefix | `{name: {startsWith: 'John'}}` |
| `endsWith` | String suffix | `{email: {endsWith: '@gmail.com'}}` |
| `matches` | Pattern match | `{text: {matches: /^[A-Z]/}}` |
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
| `allOf` | AND combinator | `{allOf: [{active: true}, {role: 'admin'}]}` |
| `anyOf` | OR combinator | `{anyOf: [{role: 'admin'}, {role: 'owner'}]}` |
**[Complete Operator Reference → ](../QUERY_OPERATORS.md )** — all operators, aliases, indexed vs in-memory support matrix, and practical examples.
2025-08-26 12:32:21 -07:00
---
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
## Aggregation Engine
Brainy's aggregation engine maintains **incremental running totals** at write time, delivering O(1) aggregate reads regardless of dataset size. Define aggregates once, and every `add()` , `update()` , and `delete()` automatically updates the running metrics.
### `defineAggregate(definition)` → `void`
Register a named aggregate for incremental computation.
```typescript
brain.defineAggregate({
name: 'monthly_spending',
source: {
type: NounType.Event,
where: { domain: 'financial', subtype: 'transaction' }
},
groupBy: [
'category',
{ field: 'date', window: 'month' } // Time-windowed dimension
],
metrics: {
total: { op: 'sum', field: 'amount' },
count: { op: 'count' },
average: { op: 'avg', field: 'amount' },
highest: { op: 'max', field: 'amount' },
2026-02-17 17:04:11 -08:00
lowest: { op: 'min', field: 'amount' },
spread: { op: 'stddev', field: 'amount' } // Welford's online algorithm
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
},
materialize: true // Optional: write results as NounType.Measurement entities
})
```
**Parameters:**
| Field | Type | Description |
|-------|------|-------------|
| `name` | `string` | Unique identifier for this aggregate |
| `source.type` | `NounType \| NounType[]` | Entity types that feed into this aggregate |
| `source.where` | `Record<string, unknown>` | Metadata filter (same syntax as `find({ where })` ) |
| `source.service` | `string` | Multi-tenancy filter |
| `groupBy` | `GroupByDimension[]` | Dimensions to group by — plain field names or `{ field, window }` for time bucketing |
2026-02-17 17:04:11 -08:00
| `metrics` | `Record<string, AggregateMetricDef>` | Named metrics with `op` (`sum` , `count` , `avg` , `min` , `max` , `stddev` , `variance` ) and optional `field` |
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
| `materialize` | `boolean \| object` | Write results as `NounType.Measurement` entities (auto-visible in OData/Sheets/SSE) |
**Time window granularities:** `'hour'` , `'day'` , `'week'` , `'month'` , `'quarter'` , `'year'` , or `{ seconds: number }` for custom intervals.
### `removeAggregate(name)` → `void`
Remove a named aggregate and clean up its state.
```typescript
brain.removeAggregate('monthly_spending')
```
### Querying Aggregates via `find()`
Aggregate results are queried through the standard `find()` method using the `aggregate` parameter:
```typescript
// Simple: query by name
const results = await brain.find({ aggregate: 'monthly_spending' })
// With filtering on group keys
const foodOnly = await brain.find({
aggregate: 'monthly_spending',
where: { category: 'food' }
})
// With sorting and pagination
const topCategories = await brain.find({
aggregate: {
name: 'monthly_spending',
orderBy: 'total',
order: 'desc',
limit: 10
}
})
// Combine find-level params (where, orderBy, limit, offset merge automatically)
const recentFood = await brain.find({
aggregate: 'monthly_spending',
where: { category: 'food' },
orderBy: 'total',
order: 'desc',
limit: 12
})
```
**Result format:** Returns `Result<T>[]` with `type: NounType.Measurement` . Each result contains:
```typescript
{
id: string, // Aggregate group ID (or materialized entity ID)
score: 1.0, // Always 1.0 for aggregates
type: NounType.Measurement,
metadata: {
__aggregate: 'monthly_spending', // Source aggregate name
category: 'food', // Group key values
date: '2024-01', // Time window bucket
total: 342.50, // Computed metrics
count: 28,
average: 12.23,
highest: 45.00,
lowest: 2.50
},
entity: Entity // Full entity structure
}
```
### How It Works
Aggregation hooks run **outside transactions** on every write operation:
- **`add()` **: If the new entity matches any aggregate's `source` filter, its values are added to the matching group's running totals.
- **`update()` **: The old entity's contribution is reversed and the new entity's contribution is applied (handles group key changes, source filter changes).
- **`delete()` **: The deleted entity's contribution is reversed from its group.
**Performance:** O(A × G × M) per write where A = matching aggregates, G = groupBy dimensions, M = metrics. For typical configurations (2-5 aggregates, 1-3 dimensions, 3-5 metrics), this is effectively O(1) — measured at **10,000 entities in 13ms** in unit tests.
**Infinite loop prevention:** Materialized `NounType.Measurement` entities (with `service: 'brainy:aggregation'` or `metadata.__aggregate` ) are automatically excluded from all aggregate source matching.
**Persistence:** Definitions and running state are persisted to storage on `flush()` /`close()` and reloaded on `init()` . Definition changes are detected via FNV-1a hashing — only changed aggregates reset their state.
**Native acceleration:** Register an `'aggregation'` provider via the plugin system to replace the TypeScript engine with a custom native implementation for higher throughput at scale.
### Financial Data Modeling
Brainy supports financial analytics through **metadata conventions** on existing NounTypes — no custom types needed:
```typescript
// Transaction = NounType.Event + financial metadata
await brain.add({
data: 'Coffee at Blue Bottle',
type: NounType.Event,
metadata: {
domain: 'financial',
subtype: 'transaction',
amount: 5.50,
currency: 'USD',
category: 'food',
date: Date.now(),
merchant: 'Blue Bottle Coffee'
}
})
// Account = NounType.Collection + financial metadata
await brain.add({
data: 'Checking Account',
type: NounType.Collection,
metadata: {
domain: 'financial',
subtype: 'account',
accountType: 'checking',
currency: 'USD',
institution: 'Chase'
}
})
// Invoice = NounType.Document + financial metadata
await brain.add({
data: 'Invoice #1234 from Acme Corp',
type: NounType.Document,
metadata: {
domain: 'financial',
subtype: 'invoice',
amount: 15000,
currency: 'USD',
status: 'pending',
dueDate: Date.UTC(2024, 2, 15),
vendor: 'Acme Corp'
}
})
```
---
2025-11-02 10:58:52 -08:00
## Relationships
### `relate(params)` → `Promise<string>`
Create a typed relationship between entities.
```typescript
const relId = await brain.relate({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
from: sourceId,
to: targetId,
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
type: VerbType.ReportsTo,
subtype: 'direct', // Optional: sub-classification
data: 'Collaborated on the research paper', // Optional: content for this edge
metadata: { // Optional: structured edge fields
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
strength: 0.9,
role: 'primary author'
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
}
2025-11-02 10:58:52 -08:00
})
```
**Parameters:**
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `from` : `string` - Source entity ID (must exist)
- `to` : `string` - Target entity ID (must exist)
2025-11-02 10:58:52 -08:00
- `type` : `VerbType` - Relationship type
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
- `subtype?` : `string` - Per-product sub-classification within the VerbType (top-level standard field, fast-path indexed). See [Subtypes & Facets ](../guides/subtypes-and-facets.md ).
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- `data?` : `any` - Content for the relationship (overrides auto-computed vector)
- `metadata?` : `object` - Structured edge fields
- `weight?` : `number` - Connection strength (0-1, default: 1.0)
- `bidirectional?` : `boolean` - Create reverse edge too (default: false)
- `confidence?` : `number` - Relationship certainty (0-1)
2025-11-02 10:58:52 -08:00
fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors
Brainy 7.30 shipped opt-in subtype enforcement; SDK 3.20.0 then registered
SDK_CORE_VOCABULARY on every consumer's brain (Event, Collection, Message,
Contract, Media, Document NounTypes). On 2026-06-08 Venue's /book flow went 500
because their brain.add({ type: NounType.Event, ... }) call sites lacked
subtype. An audit of Brainy's OWN source revealed 14 HIGH-risk internal write
paths that also omit subtype — any consumer running the same vocabulary would
have hit Brainy's infrastructure paths next. 7.30.1 closes both gaps before
8.0 makes strict mode the default.
Additive across the board. Zero behavior change for consumers not using strict
mode. Every change is JS-side — Cortex needs no work for 7.30.1.
NEW — brain.audit() diagnostic
- Read-only method walking storage.getNouns() / getVerbs() pagination
- Returns { entitiesWithoutSubtype: { type: count }, relationshipsWithoutSubtype,
total, scanned, recommendation }
- VFS infrastructure entities excluded by default (they bypass enforcement via
isVFSEntity marker); pass { includeVFS: true } to surface them
- The companion to migrateField (7.x) and fillSubtypes (8.0): tells consumers
exactly what would break under strict enforcement, deterministically
NEW — Improved enforcement error messages
- Caller's source location extracted from Error().stack so users see their own
call site, not a Brainy internal frame
- Specific guidance branches: registered vocabulary → "Pass one of: a, b, c";
brain-wide strict mode → mentions the except clause; otherwise → registration
recipe via brain.requireSubtype()
- Documentation link to the canonical migration recipe
- Same shape for noun and verb enforcement
NEW — CLI --subtype flag
- brainy add and brainy relate gain -s/--subtype <value>
- Defaults to 'cli-add' / 'cli-relate' so the CLI works against strict-mode
brains without the user needing to know the vocabulary in advance
INTERNAL — every Brainy write path now sets subtype
- VFS Contains edges (5 sites at lines 503/905/1694/1772/1886) → 'vfs-contains'
- VFS symlink entity → 'vfs-symlink' (NEW — distinct from 'vfs-file')
- VFS copy-file → preserves source subtype, falls back to 'vfs-file'
- VFS symlink also adopts the isVFSEntity infrastructure marker so it bypasses
enforcement in strict mode
- Aggregation materializer (Measurement entities) → 'materialized-aggregate'
- ImportCoordinator (3 sites): document → 'import-source'; entities →
options.defaultSubtype ?? 'imported'; placeholder → 'import-placeholder'
- SmartImportOrchestrator (4 entity sites + 2 batch relate sites): same
precedence (extractor → options.defaultSubtype → 'imported')
- EntityDeduplicator → candidate.subtype ?? 'imported'
- UniversalImportAPI → extractor → 'extracted' for both entities and relations
- NeuralImport → adds defaultSubtype to NeuralImportOptions; precedence same
- GoogleSheetsIntegration → request body 'subtype' ?? 'imported-from-sheets'
- ODataIntegration → request body 'Subtype' ?? 'imported-from-odata'
- MCP client message storage → 'mcp-message' (also fixes pre-existing missing
data field and missing type by aliasing from the prior text field)
Side-effect fix: storage.getNouns() paginated now surfaces subtype to top-level
- Single-noun getNoun() already did this in 7.30; the paginated path was missed
- Without this fix brain.audit() saw missing subtype on entities that actually
had one (caught by the strict-mode self-test before release)
NEW — tests/integration/strict-mode-self-test.test.ts (13 tests)
- Creates a brain under the exact SDK_CORE_VOCABULARY shape Venue hit + brain-
wide strict mode
- Exercises every internal Brainy path: VFS root + mkdir + writeFile + cp + mv
+ ln + symlink; aggregation engine; audit diagnostic with includeVFS toggle
- Validates error message UX: caller location, vocabulary guidance, brain-wide
strict mode guidance, off-vocabulary value reporting
Docs
- New "Strict mode in practice" section in docs/guides/subtypes-and-facets.md
covering the SDK_CORE_VOCABULARY pattern, 4-step migration recipe
(audit → migrateField → hand-fix → re-audit), the Brainy-internal label
reference table, and an 8.0 forward-look on fillSubtypes()
- docs/api/README.md: new audit() entry, strict-mode tips on add() and relate()
- RELEASES.md: full 7.30.1 entry
Cortex parity (forward-looking, not blocking 7.30.1)
- 6th open question added to .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md: native
fast path for audit() and fillSubtypes() via column-store null-subtype
bitmap for billion-scale brains
- Cortex should add a parity test mirroring strict-mode-self-test.test.ts
against their native paths to catch any latent bug where native writes
bypass JS validation
- Brainy-internal subtype labels become a documented part of the 8.0 contract
(useful for Cortex telemetry surfacing Brainy-managed infrastructure %)
Verification
- npx tsc --noEmit: clean
- npm test: 1468/1468 unit
- 7.29 noun integration suite: 26/26 (no regression)
- 7.30 verb subtype + enforcement integration suite: 30/30 (no regression)
- New strict-mode-self-test integration suite: 13/13
- npm run build: clean
- Closed-source product reference audit: clean
Addresses VE-SUBTYPE-MIGRATION (Venue's reported request) and ships internal
labels Venue did NOT ask for but that would have broken them next under their
own vocabulary registration.
2026-06-08 11:31:47 -07:00
> **Strict-mode tip:** same as `add()` — if a vocabulary is registered for your `type`, pass a matching `subtype`. Run `await brain.audit()` first to surface pre-existing gaps.
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<string>` - Relationship ID
---
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
### `updateRelation(params)` → `Promise<void>`
Update an existing relationship. Mirror of `update()` for verbs — closed a long-standing gap (verbs had no update path before 7.30).
```typescript
// Change the subtype on an existing relationship
await brain.updateRelation({ id: relId, subtype: 'dotted-line' })
// Update weight + confidence
await brain.updateRelation({ id: relId, weight: 0.7, confidence: 0.9 })
// Change verb type (re-indexes in graph adjacency, id preserved)
await brain.updateRelation({ id: relId, type: VerbType.WorksWith })
```
**Parameters:**
- `id` : `string` - Relationship ID (required)
- `type?` : `VerbType` - Change verb type (re-indexes in graph adjacency)
- `subtype?` : `string` - Change sub-classification (omit to preserve existing)
- `weight?` : `number` - New weight (0-1)
- `confidence?` : `number` - New confidence (0-1)
- `data?` : `any` - New content
- `metadata?` : `object` - Metadata to merge (or replace with `merge: false` )
- `merge?` : `boolean` - Merge or replace metadata (default: true)
**Returns:** `Promise<void>`
---
2025-11-02 10:58:52 -08:00
### `getRelations(params)` → `Promise<Relation[]>`
Get relationships for an entity.
2025-08-26 12:32:21 -07:00
```typescript
2025-11-02 10:58:52 -08:00
// Get all relationships FROM an entity
const outgoing = await brain.getRelations({ from: entityId })
// Get all relationships TO an entity
const incoming = await brain.getRelations({ to: entityId })
// Filter by type
const related = await brain.getRelations({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
from: entityId,
type: VerbType.Contains
2025-08-26 12:32:21 -07:00
})
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
// Filter by subtype (fast path, column-store hit)
const direct = await brain.getRelations({
from: entityId,
type: VerbType.ReportsTo,
subtype: 'direct'
})
// Set membership on subtype
const all = await brain.getRelations({
from: entityId,
type: VerbType.ReportsTo,
subtype: ['direct', 'dotted-line']
})
2025-11-02 10:58:52 -08:00
```
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**Parameters:**
- `from?` : `string` - Source entity ID
- `to?` : `string` - Target entity ID
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
- `type?` : `VerbType | VerbType[]` - Filter by relationship type
- `subtype?` : `string | string[]` - Filter by VerbType subtype (top-level standard field, fast path)
- `service?` : `string` - Multi-tenancy filter
- `limit?` : `number` - Pagination limit (default: 100)
- `offset?` : `number` - Pagination offset
2025-08-26 12:32:21 -07:00
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
**Returns:** `Promise<Relation[]>` - Matching relationships (each with `subtype` at top level when set)
2025-11-02 10:58:52 -08:00
---
## Batch Operations
### `addMany(params)` → `Promise<BatchResult<string>>`
Add multiple entities in one operation.
```typescript
const result = await brain.addMany({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
items: [
{ data: 'Entity 1', type: NounType.Document },
{ data: 'Entity 2', type: NounType.Concept }
]
2025-11-02 10:58:52 -08:00
})
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(result.successful) // Array of IDs
console.log(result.failed) // Array of errors
2025-11-02 10:58:52 -08:00
```
**Returns:** `Promise<BatchResult<string>>` - Success/failure results
---
### `deleteMany(params)` → `Promise<BatchResult<string>>`
Delete multiple entities.
```typescript
const result = await brain.deleteMany({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
ids: [id1, id2, id3]
2025-11-02 10:58:52 -08:00
})
```
---
### `updateMany(params)` → `Promise<BatchResult<string>>`
Update multiple entities.
```typescript
const result = await brain.updateMany({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
updates: [
{ id: id1, metadata: { updated: true } },
{ id: id2, data: 'New content' }
]
2025-11-02 10:58:52 -08:00
})
```
---
### `relateMany(params)` → `Promise<string[]>`
Create multiple relationships.
```typescript
const ids = await brain.relateMany({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
relations: [
{ from: id1, to: id2, type: VerbType.RelatedTo },
{ from: id1, to: id3, type: VerbType.Contains }
]
2025-11-02 10:58:52 -08:00
})
```
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
## Branch Management
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
Git-style branching with Snowflake-style copy-on-write.
2025-11-02 10:58:52 -08:00
### `fork(branch?, options?)` → `Promise<Brainy>`
Create an instant fork (< 100ms ) with full isolation .
```typescript
// Create a fork
const experiment = await brain.fork('test-feature')
// Make changes safely in isolation
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
await experiment.add({ data: 'Test entity', type: NounType.Document })
2025-11-02 10:58:52 -08:00
await experiment.update({ id: someId, metadata: { modified: true } })
// Parent is unaffected!
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
const parentData = await brain.find({}) // Original data unchanged
2025-11-02 10:58:52 -08:00
```
**Parameters:**
- `branch?` : `string` - Branch name (auto-generated if omitted)
- `options?` : `object`
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `description?` : `string` - Branch description
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<Brainy>` - New Brainy instance on forked branch
**How it works:** Snowflake-style COW shares HNSW index, copies only modified nodes (10-20% memory overhead).
---
### `checkout(branch)` → `Promise<void>`
Switch to a different branch.
```typescript
await brain.checkout('main')
await brain.checkout('test-feature')
```
**Parameters:**
- `branch` : `string` - Branch name
---
### `listBranches()` → `Promise<string[]>`
List all branches.
```typescript
const branches = await brain.listBranches()
// ['main', 'test-feature', 'experiment-2']
```
---
### `getCurrentBranch()` → `Promise<string>`
Get current branch name.
```typescript
const current = await brain.getCurrentBranch()
// 'main'
```
---
### `commit(options?)` → `Promise<string>`
Create a commit snapshot.
```typescript
const commitId = await brain.commit({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
message: 'Add new features',
author: 'dev@example .com',
metadata: { ticket: 'PROJ-123' }
2025-11-02 10:58:52 -08:00
})
```
**Parameters:**
- `message?` : `string` - Commit message
- `author?` : `string` - Author email
- `metadata?` : `object` - Additional commit metadata
**Returns:** `Promise<string>` - Commit ID
---
### `deleteBranch(branch)` → `Promise<void>`
Delete a branch (cannot delete 'main').
```typescript
await brain.deleteBranch('old-experiment')
```
---
### `getHistory(options?)` → `Promise<Commit[]>`
Get commit history.
```typescript
const history = await brain.getHistory({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
branch: 'main',
limit: 10
2025-08-26 12:32:21 -07:00
})
```
2025-11-02 10:58:52 -08:00
---
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
### `asOf(commitId, options?)` → `Promise<Brainy>`
Create a read-only snapshot at a specific commit for time-travel queries.
```typescript
// Get commit ID from history
const commits = await brain.getHistory({ limit: 1 })
const commitId = commits[0].id
// Create snapshot (lazy-loading, no eager data loading)
const snapshot = await brain.asOf(commitId, {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
cacheSize: 10000 // LRU cache size (default: 10000)
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
})
// Query historical state - full Triple Intelligence works!
const results = await snapshot.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'AI research',
where: { category: 'technology' }
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
})
// Get historical relationships
const related = await snapshot.getRelated(entityId, { depth: 2 })
// MUST close when done to free memory
await snapshot.close()
```
**Parameters:**
- `commitId` : `string` - Commit hash to snapshot from
- `options?` : `object`
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `cacheSize?` : `number` - LRU cache size for lazy-loading (default: 10000)
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
**Returns:** `Promise<Brainy>` - Read-only Brainy instance with historical state
**Features:**
- **Lazy-Loading** - Loads entities on-demand, not eagerly
- **Bounded Memory** - LRU cache prevents memory bloat
- **Full Query Support** - All find(), getRelated(), etc. work on historical data
- **Read-Only** - Prevents accidental modifications to history
**Important:** Always call `snapshot.close()` when done to release resources.
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
## Entity Versioning
2025-11-04 11:19:02 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
Git-style versioning for individual entities with content-addressable storage.
2025-11-04 11:19:02 -08:00
### Overview
Entity Versioning provides time-travel and history tracking for individual entities:
- **Content-Addressable Storage** - Deduplication via SHA-256 hashing
- **Zero-Config** - Lazy initialization, uses existing indexes
- **Branch-Isolated** - Versions isolated per branch
- **Selective Auto-Versioning** - Optional augmentation for automatic version creation
- **Production-Scale** - Designed for billions of entities
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- **VFS File Support** - Full versioning for VFS files with actual blob content
2025-11-04 11:19:02 -08:00
---
### `versions.save(entityId, options?)` → `Promise<EntityVersion>`
Save a new version of an entity.
```typescript
// Save version with tag
const version = await brain.versions.save('user-123', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
tag: 'v1.0',
description: 'Initial user profile',
metadata: { author: 'dev@example .com' }
2025-11-04 11:19:02 -08:00
})
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(version.version) // 1
console.log(version.contentHash) // SHA-256 hash
console.log(version.createdAt) // Timestamp
2025-11-04 11:19:02 -08:00
```
**Parameters:**
- `entityId` : `string` - Entity ID to version
- `options?` : `object`
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `tag?` : `string` - Version tag (e.g., 'v1.0', 'beta')
- `description?` : `string` - Version description
- `metadata?` : `object` - Additional version metadata
2025-11-04 11:19:02 -08:00
**Returns:** `Promise<EntityVersion>` - Created version
**Features:**
- Automatic deduplication (identical content = same version)
- Sequential version numbering (1, 2, 3, ...)
- Content-addressable storage (SHA-256)
---
### `versions.list(entityId, options?)` → `Promise<EntityVersion[]>`
List all versions of an entity.
```typescript
const versions = await brain.versions.list('user-123', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
limit: 10,
offset: 0
2025-11-04 11:19:02 -08:00
})
versions.forEach(v => {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(`Version ${v.version}: ${v.tag} - ${v.description}` )
2025-11-04 11:19:02 -08:00
})
```
**Parameters:**
- `entityId` : `string` - Entity ID
- `options?` : `object`
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `limit?` : `number` - Max versions to return
- `offset?` : `number` - Skip versions
2025-11-04 11:19:02 -08:00
**Returns:** `Promise<EntityVersion[]>` - Versions (newest first)
---
### `versions.restore(entityId, versionOrTag)` → `Promise<void>`
Restore entity to a previous version.
```typescript
// Restore by version number
await brain.versions.restore('user-123', 1)
// Restore by tag
await brain.versions.restore('user-123', 'beta')
```
**Parameters:**
- `entityId` : `string` - Entity ID
- `versionOrTag` : `number | string` - Version number or tag
---
### `versions.compare(entityId, version1, version2)` → `Promise<VersionDiff>`
Compare two versions.
```typescript
const diff = await brain.versions.compare('user-123', 1, 2)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(diff.totalChanges) // Total changes
console.log(diff.modified) // Modified fields
console.log(diff.added) // Added fields
console.log(diff.removed) // Removed fields
2025-11-04 11:19:02 -08:00
// Check specific changes
const nameChange = diff.modified.find(c => c.path === 'metadata.name')
console.log(`${nameChange.oldValue} → ${nameChange.newValue}` )
```
**Returns:** `Promise<VersionDiff>` - Detailed diff with field-level changes
---
### `versions.getContent(entityId, versionOrTag)` → `Promise<EntitySnapshot>`
Get version content without restoring.
```typescript
// View old version without changing current state
const v1Content = await brain.versions.getContent('user-123', 1)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(v1Content.metadata.name) // Old name
2025-11-04 11:19:02 -08:00
// Current state unchanged
const current = await brain.get('user-123')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(current.metadata.name) // Current name
2025-11-04 11:19:02 -08:00
```
---
### `versions.undo(entityId)` → `Promise<void>`
Undo to previous version (shorthand for restore to latest-1).
```typescript
// Make a bad change
await brain.update('user-123', { status: 'deleted' })
// Undo immediately
await brain.versions.undo('user-123')
```
**Alias:** `versions.revert(entityId)`
---
### `versions.prune(entityId, options)` → `Promise<PruneResult>`
Clean up old versions.
```typescript
const result = await brain.versions.prune('user-123', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
keepRecent: 10, // Keep 10 most recent
keepTagged: true, // Always keep tagged versions
olderThan: Date.now() - 30 * 24 * 60 * 60 * 1000 // Older than 30 days
2025-11-04 11:19:02 -08:00
})
console.log(`Deleted ${result.deleted}, kept ${result.kept}` )
```
**Parameters:**
- `keepRecent?` : `number` - Keep N most recent versions
- `keepTagged?` : `boolean` - Always keep tagged versions (default: true)
- `olderThan?` : `number` - Only prune versions older than timestamp
---
### `versions.getLatest(entityId)` → `Promise<EntityVersion | null>`
Get latest version.
```typescript
const latest = await brain.versions.getLatest('user-123')
if (latest) {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(`Latest: v${latest.version} (${latest.tag})` )
2025-11-04 11:19:02 -08:00
}
```
---
### `versions.getVersionByTag(entityId, tag)` → `Promise<EntityVersion | null>`
Get version by tag.
```typescript
const beta = await brain.versions.getVersionByTag('user-123', 'beta')
```
---
### `versions.count(entityId)` → `Promise<number>`
Count versions for an entity.
```typescript
const count = await brain.versions.count('user-123')
console.log(`${count} versions saved` )
```
---
### `versions.hasVersions(entityId)` → `Promise<boolean>`
Check if entity has versions.
```typescript
if (await brain.versions.hasVersions('user-123')) {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log('Entity has version history')
2025-11-04 11:19:02 -08:00
}
```
---
### Auto-Versioning Augmentation
Automatically create versions on entity updates.
```typescript
import { VersioningAugmentation } from '@soulcraft/brainy '
// Configure auto-versioning
const versioning = new VersioningAugmentation({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
enabled: true,
onUpdate: true, // Version on update()
onDelete: false, // Don't version on delete
entities: ['user-*'], // Only version users
excludeEntities: ['temp-*'],
excludeTypes: ['temporary'],
keepRecent: 50, // Auto-prune old versions
keepTagged: true
2025-11-04 11:19:02 -08:00
})
// Apply augmentation
brain.augment(versioning)
// Now updates auto-create versions
await brain.update('user-123', { name: 'New Name' })
// Version automatically created!
const versions = await brain.versions.list('user-123')
console.log(`Auto-created version: ${versions[0].version}` )
```
**Configuration:**
- `enabled` : `boolean` - Enable/disable augmentation
- `onUpdate` : `boolean` - Version on entity updates
- `onDelete` : `boolean` - Version before deletion
- `entities` : `string[]` - Entity ID patterns (glob-style)
- `excludeEntities` : `string[]` - Exclusion patterns
- `types` : `string[]` - Entity types to version
- `excludeTypes` : `string[]` - Types to exclude
- `keepRecent` : `number` - Auto-prune to keep N versions
- `keepTagged` : `boolean` - Always keep tagged versions
**Pattern Matching:**
- `['*']` - All entities
- `['user-*']` - All IDs starting with "user-"
- `['*-prod']` - All IDs ending with "-prod"
- `['user-*', 'account-*']` - Multiple patterns
---
### Branch Isolation
Versions are isolated per branch.
```typescript
// Save version on main
await brain.versions.save('doc-1', { tag: 'main-v1' })
// Fork and create version
const feature = await brain.fork('feature')
await feature.update('doc-1', { content: 'Feature update' })
await feature.versions.save('doc-1', { tag: 'feature-v1' })
// Versions are isolated
const mainVersions = await brain.versions.list('doc-1')
const featureVersions = await feature.versions.list('doc-1')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(mainVersions.length !== featureVersions.length) // true
2025-11-04 11:19:02 -08:00
```
---
### Architecture
**Content-Addressable Storage:**
- SHA-256 hashing for deduplication
- Identical content = single storage blob
- Efficient for entities with few changes
**Metadata Indexing:**
- Leverages existing MetadataIndexManager
- Fast lookups by entity ID
- Version number indexing
**Storage Structure:**
```
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
_version:{entityId}:{versionNum}:{branch} // Version metadata
_version_blob:{contentHash} // Content blob (deduplicated)
2025-11-04 11:19:02 -08:00
```
**Performance:**
- Version save: O(1) if duplicate, O(log N) for index update
- Version list: O(K) where K = version count
- Version restore: O(log N) lookup + O(1) restore
- Pruning: O(K) where K = versions pruned
---
### Examples
#### Basic Versioning Workflow
```typescript
// Create entity
await brain.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'User profile',
id: 'user-123',
type: 'user',
metadata: { name: 'Alice', email: 'alice@example .com' }
2025-11-04 11:19:02 -08:00
})
// Save v1
await brain.versions.save('user-123', { tag: 'v1.0' })
// Make changes
await brain.update('user-123', { name: 'Alice Smith' })
// Save v2
await brain.versions.save('user-123', { tag: 'v2.0' })
// Compare versions
const diff = await brain.versions.compare('user-123', 1, 2)
// Restore to v1 if needed
await brain.versions.restore('user-123', 'v1.0')
```
#### Release Management
```typescript
// Development workflow
await brain.update('app-config', { version: '1.0.0-alpha' })
await brain.versions.save('app-config', { tag: 'alpha' })
await brain.update('app-config', { version: '1.0.0-beta' })
await brain.versions.save('app-config', { tag: 'beta' })
await brain.update('app-config', { version: '1.0.0' })
await brain.versions.save('app-config', { tag: 'release' })
// Rollback to beta if issues found
await brain.versions.restore('app-config', 'beta')
```
#### Audit Trail
```typescript
// Track all changes
const versioning = new VersioningAugmentation({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
enabled: true,
onUpdate: true,
entities: ['audit-*'],
keepRecent: 100 // Keep 100 versions for audit
2025-11-04 11:19:02 -08:00
})
brain.augment(versioning)
// All updates now tracked
await brain.update('audit-record-1', { status: 'modified' })
await brain.update('audit-record-1', { status: 'approved' })
// View complete history
const versions = await brain.versions.list('audit-record-1')
versions.forEach(v => {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(`${v.createdAt}: ${v.description}` )
2025-11-04 11:19:02 -08:00
})
```
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
#### VFS File Versioning
2025-12-09 16:13:45 -08:00
```typescript
// VFS files can be versioned with actual blob content
await brain.vfs.writeFile('/docs/readme.md', 'Version 1 content')
// Get the file's entity ID
const stat = await brain.vfs.stat('/docs/readme.md')
// Save version 1
await brain.versions.save(stat.entityId, { tag: 'v1', description: 'Initial draft' })
// Modify the file
await brain.vfs.writeFile('/docs/readme.md', 'Version 2 - updated content')
// Save version 2
await brain.versions.save(stat.entityId, { tag: 'v2', description: 'Updated docs' })
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Compare versions - content is DIFFERENT
2025-12-09 16:13:45 -08:00
const v1 = await brain.versions.getContent(stat.entityId, 1)
const v2 = await brain.versions.getContent(stat.entityId, 2)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(v1.data !== v2.data) // true
2025-12-09 16:13:45 -08:00
// Restore to v1 - writes content back to blob storage
await brain.versions.restore(stat.entityId, 'v1')
// File is now back to v1
const content = await brain.vfs.readFile('/docs/readme.md')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(content.toString()) // 'Version 1 content'
2025-12-09 16:13:45 -08:00
```
2025-11-04 11:19:02 -08:00
---
**[📖 Complete Versioning Guide → ](../features/entity-versioning.md )**
---
2025-11-02 10:58:52 -08:00
## Virtual Filesystem (VFS)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
Access via `brain.vfs` (property, not method). Auto-initialized during `brain.init()` .
2025-11-02 10:58:52 -08:00
2025-11-04 11:19:02 -08:00
### Filtering VFS Entities
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
All VFS entities (files/folders) have `metadata.isVFSEntity: true` set automatically.
2025-11-04 11:19:02 -08:00
Use this to filter VFS entities from semantic search results:
```typescript
// Exclude VFS entities from semantic search
const semanticOnly = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'artificial intelligence',
where: {
isVFSEntity: { notEquals: true } // Only semantic entities
}
2025-11-04 11:19:02 -08:00
})
// Or filter to ONLY VFS entities
const vfsOnly = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
where: {
isVFSEntity: { equals: true } // Only VFS files/folders
}
2025-11-04 11:19:02 -08:00
})
// Check if an entity is a VFS entity
if (entity.metadata.isVFSEntity === true) {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log('This is a VFS file or folder')
2025-11-04 11:19:02 -08:00
}
```
**Why this matters:** Without filtering, VFS files/folders can appear in concept explorers and semantic search results where they don't belong.
---
2025-11-02 10:58:52 -08:00
### Basic File Operations
#### `vfs.readFile(path, options?)` → `Promise<Buffer>`
Read file content.
2025-08-26 12:32:21 -07:00
```typescript
2025-11-02 10:58:52 -08:00
const content = await brain.vfs.readFile('/docs/README.md')
console.log(content.toString())
```
---
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
#### `vfs.writeFile(path, data, options?)` → `Promise<void>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
Write file content.
```typescript
await brain.vfs.writeFile('/docs/README.md', 'New content', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
encoding: 'utf-8'
2025-08-26 12:32:21 -07:00
})
```
2025-11-02 10:58:52 -08:00
---
#### `vfs.unlink(path)` → `Promise<void>`
Delete a file.
2025-08-26 12:32:21 -07:00
```typescript
2025-11-02 10:58:52 -08:00
await brain.vfs.unlink('/docs/old-file.md')
```
---
### Directory Operations
#### `vfs.mkdir(path, options?)` → `Promise<void>`
Create directory.
```typescript
await brain.vfs.mkdir('/projects/new-app', { recursive: true })
```
---
#### `vfs.readdir(path, options?)` → `Promise<string[] | Dirent[]>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
List directory contents.
```typescript
const files = await brain.vfs.readdir('/projects')
// With file types
const entries = await brain.vfs.readdir('/projects', { withFileTypes: true })
entries.forEach(entry => {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(entry.name, entry.isDirectory() ? 'DIR' : 'FILE')
2025-08-26 12:32:21 -07:00
})
2025-11-02 10:58:52 -08:00
```
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
---
#### `vfs.rmdir(path, options?)` → `Promise<void>`
Remove directory.
```typescript
await brain.vfs.rmdir('/old-project', { recursive: true })
```
---
#### `vfs.stat(path)` → `Promise<Stats>`
Get file/directory stats.
```typescript
const stats = await brain.vfs.stat('/docs/README.md')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(stats.size) // File size
console.log(stats.mtime) // Modified time
console.log(stats.isDirectory()) // Is directory?
2025-11-02 10:58:52 -08:00
```
---
### Semantic Operations
#### `vfs.search(query, options?)` → `Promise<SearchResult[]>`
Semantic file search.
```typescript
const results = await brain.vfs.search('React components with hooks', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
path: '/src',
limit: 10
2025-11-02 10:58:52 -08:00
})
```
---
#### `vfs.findSimilar(path, options?)` → `Promise<SearchResult[]>`
Find similar files.
```typescript
const similar = await brain.vfs.findSimilar('/src/App.tsx', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
limit: 5,
threshold: 0.7
2025-11-02 10:58:52 -08:00
})
```
---
### Tree Operations
#### `vfs.getTreeStructure(path, options?)` → `Promise<TreeNode>`
Get directory tree (prevents infinite recursion).
```typescript
const tree = await brain.vfs.getTreeStructure('/projects', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
maxDepth: 3
2025-11-02 10:58:52 -08:00
})
```
---
#### `vfs.getDescendants(path, options?)` → `Promise<VFSEntity[]>`
Get all descendants with optional filtering.
```typescript
const files = await brain.vfs.getDescendants('/src', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
filter: (entity) => entity.name.endsWith('.tsx')
2025-08-26 12:32:21 -07:00
})
```
---
2025-11-02 10:58:52 -08:00
### Metadata & Relationships
#### `vfs.getMetadata(path)` → `Promise<Metadata>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
Get file metadata.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
```typescript
const meta = await brain.vfs.getMetadata('/src/App.tsx')
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(meta.todos) // Extracted TODOs
console.log(meta.tags) // Tags
2025-11-02 10:58:52 -08:00
```
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
---
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
#### `vfs.getRelationships(path)` → `Promise<Relation[]>`
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
Get file relationships.
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
```typescript
const rels = await brain.vfs.getRelationships('/src/App.tsx')
// Returns: imports, references, dependencies
```
2025-08-26 12:32:21 -07:00
---
2025-11-02 10:58:52 -08:00
#### `vfs.getTodos(path)` → `Promise<Todo[]>`
Get TODOs from a file.
```typescript
const todos = await brain.vfs.getTodos('/src/App.tsx')
```
---
#### `vfs.getAllTodos(path?)` → `Promise<Todo[]>`
Get all TODOs from directory tree.
```typescript
const allTodos = await brain.vfs.getAllTodos('/src')
```
---
### Project Analysis
#### `vfs.getProjectStats(path?)` → `Promise<Stats>`
Get project statistics.
```typescript
const stats = await brain.vfs.getProjectStats('/projects/my-app')
console.log(stats.fileCount)
console.log(stats.totalSize)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(stats.fileTypes) // Breakdown by extension
2025-11-02 10:58:52 -08:00
```
---
#### `vfs.searchEntities(query)` → `Promise<VFSEntity[]>`
Search for VFS entities by metadata.
```typescript
const tsxFiles = await brain.vfs.searchEntities({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
type: 'file',
extension: '.tsx'
2025-11-02 10:58:52 -08:00
})
```
---
**[📖 Complete VFS Documentation → ](../vfs/QUICK_START.md )**
---
## Neural API
Access advanced AI features via `brain.neural()` (method that returns NeuralAPI instance).
### `neural().similar(a, b, options?)` → `Promise<number | SimilarityResult>`
Calculate semantic similarity.
```typescript
// Simple similarity score
const score = await brain.neural().similar(
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
'renewable energy',
'sustainable power'
) // 0.87
2025-11-02 10:58:52 -08:00
// Detailed result
const result = await brain.neural().similar('text1', 'text2', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
detailed: true
2025-11-02 10:58:52 -08:00
})
console.log(result.score)
console.log(result.explanation)
```
---
### `neural().clusters(input?, options?)` → `Promise<Cluster[]>`
Automatic clustering.
```typescript
const clusters = await brain.neural().clusters({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
algorithm: 'kmeans',
k: 5,
minSize: 3
2025-11-02 10:58:52 -08:00
})
clusters.forEach(cluster => {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(cluster.label)
console.log(cluster.items)
console.log(cluster.centroid)
2025-11-02 10:58:52 -08:00
})
```
---
### `neural().neighbors(id, options?)` → `Promise<Neighbor[]>`
Find k-nearest neighbors.
```typescript
const neighbors = await brain.neural().neighbors(entityId, {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
k: 10,
threshold: 0.7
2025-11-02 10:58:52 -08:00
})
```
---
### `neural().outliers(threshold?)` → `Promise<string[]>`
Detect outlier entities.
```typescript
const outliers = await brain.neural().outliers(0.3)
// Returns entity IDs that are outliers
```
---
### `neural().visualize(options?)` → `Promise<VizData>`
Generate visualization data.
```typescript
const vizData = await brain.neural().visualize({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
maxNodes: 100,
dimensions: 3,
algorithm: 'force',
includeEdges: true
2025-11-02 10:58:52 -08:00
})
// Use with D3.js, Cytoscape, GraphML tools
```
---
### Performance Methods
#### `neural().clusterFast(options)` → `Promise<Cluster[]>`
Fast clustering for large datasets.
```typescript
const clusters = await brain.neural().clusterFast({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
k: 10,
maxIterations: 50
2025-11-02 10:58:52 -08:00
})
```
---
#### `neural().clusterLarge(options)` → `Promise<Cluster[]>`
Streaming clustering for very large datasets.
```typescript
const clusters = await brain.neural().clusterLarge({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
k: 20,
batchSize: 1000
2025-11-02 10:58:52 -08:00
})
```
---
## Import & Export
### `import(source, options?)` → `Promise<ImportResult>`
Smart import with auto-detection (CSV, Excel, PDF, JSON, URLs).
```typescript
// CSV import
await brain.import('data.csv', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
format: 'csv',
createEntities: true
2025-11-02 10:58:52 -08:00
})
// Excel import
await brain.import('sales.xlsx', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
format: 'excel',
sheets: ['Q1', 'Q2']
2025-11-02 10:58:52 -08:00
})
// PDF import
await brain.import('research.pdf', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
format: 'pdf',
extractTables: true
2025-11-02 10:58:52 -08:00
})
// URL import
await brain.import('https://api.example.com/data.json')
```
**Parameters:**
- `source` : `string | Buffer | object` - File path, URL, buffer, or object
- `options?` : Import configuration
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
- `format?` : `'csv' | 'excel' | 'pdf' | 'json'` - Auto-detected if omitted
- `createEntities?` : `boolean` - Create entities from rows
- `sheets?` : `string[]` - Excel sheets to import
- `extractTables?` : `boolean` - Extract tables from PDF
2025-11-02 10:58:52 -08:00
**Returns:** `Promise<ImportResult>` - Import statistics
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
**Note:** Import always uses the current branch.
2025-11-02 10:58:52 -08:00
**[📖 Complete Import Guide → ](../guides/import-anything.md )**
---
feat: portable graph export()/import() (BackupData v1) on brain.data()
brain.data() now serializes part or all of a brain to a versioned, portable
JSON document (BackupData) and restores it — the path for partial backups,
cross-environment moves, and 7.x→8.0 migration.
- export(selector?, options?): select by ids, collection (+transitive Contains),
connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural
and predicate selectors compose. Options: includeVectors, includeContent (VFS
blobs), includeSystem, edges ('induced'|'incident'|'none').
- import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip';
reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds
to clone a subgraph under fresh ids.
- BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/
relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence,
weight, service) top-level; metadata custom-only. Current-state, no generations.
- Replaces the prior flat-entity export() (dropped relations) with a graph-complete
document. Distinct from brain.import(file) ingestion, which is unchanged.
- Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/
ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export`
now writes a BackupData document.
Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00
### Export & Import (portable backup)
`brain.data()` exposes a portable graph backup/restore API. `export(selector?, options?)`
serializes part or all of the graph to a versioned, portable `BackupData` document;
`import(backup, options?)` restores it (dedup-by-id merge by default, re-embedding when
vectors are absent).
2025-11-02 10:58:52 -08:00
```typescript
feat: portable graph export()/import() (BackupData v1) on brain.data()
brain.data() now serializes part or all of a brain to a versioned, portable
JSON document (BackupData) and restores it — the path for partial backups,
cross-environment moves, and 7.x→8.0 migration.
- export(selector?, options?): select by ids, collection (+transitive Contains),
connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural
and predicate selectors compose. Options: includeVectors, includeContent (VFS
blobs), includeSystem, edges ('induced'|'incident'|'none').
- import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip';
reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds
to clone a subgraph under fresh ids.
- BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/
relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence,
weight, service) top-level; metadata custom-only. Current-state, no generations.
- Replaces the prior flat-entity export() (dropped relations) with a graph-complete
document. Distinct from brain.import(file) ingestion, which is unchanged.
- Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/
ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export`
now writes a BackupData document.
Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00
const data = await brain.data()
// Whole brain → a portable BackupData document
const backup = await data.export()
2025-11-02 10:58:52 -08:00
feat: portable graph export()/import() (BackupData v1) on brain.data()
brain.data() now serializes part or all of a brain to a versioned, portable
JSON document (BackupData) and restores it — the path for partial backups,
cross-environment moves, and 7.x→8.0 migration.
- export(selector?, options?): select by ids, collection (+transitive Contains),
connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural
and predicate selectors compose. Options: includeVectors, includeContent (VFS
blobs), includeSystem, edges ('induced'|'incident'|'none').
- import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip';
reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds
to clone a subgraph under fresh ids.
- BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/
relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence,
weight, service) top-level; metadata custom-only. Current-state, no generations.
- Replaces the prior flat-entity export() (dropped relations) with a graph-complete
document. Distinct from brain.import(file) ingestion, which is unchanged.
- Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/
ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export`
now writes a BackupData document.
Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00
// Just one workbench's members, with vectors, then restore elsewhere (merge by id)
const subset = await data.export({ ids }, { includeVectors: true })
await otherBrain.data().then(d => d.import(subset, { onConflict: 'merge' }))
2025-11-02 10:58:52 -08:00
feat: portable graph export()/import() (BackupData v1) on brain.data()
brain.data() now serializes part or all of a brain to a versioned, portable
JSON document (BackupData) and restores it — the path for partial backups,
cross-environment moves, and 7.x→8.0 migration.
- export(selector?, options?): select by ids, collection (+transitive Contains),
connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural
and predicate selectors compose. Options: includeVectors, includeContent (VFS
blobs), includeSystem, edges ('induced'|'incident'|'none').
- import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip';
reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds
to clone a subgraph under fresh ids.
- BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/
relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence,
weight, service) top-level; metadata custom-only. Current-state, no generations.
- Replaces the prior flat-entity export() (dropped relations) with a graph-complete
document. Distinct from brain.import(file) ingestion, which is unchanged.
- Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/
ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export`
now writes a BackupData document.
Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00
// A collection + its children; a connected neighbourhood; a VFS subtree (+ bytes)
await data.export({ collection: collectionId })
await data.export({ connected: { from: id, depth: 2 } })
await data.export({ vfsPath: '/docs' }, { includeContent: true })
2025-11-02 10:58:52 -08:00
```
feat: portable graph export()/import() (BackupData v1) on brain.data()
brain.data() now serializes part or all of a brain to a versioned, portable
JSON document (BackupData) and restores it — the path for partial backups,
cross-environment moves, and 7.x→8.0 migration.
- export(selector?, options?): select by ids, collection (+transitive Contains),
connected neighbourhood, vfsPath subtree, predicate, or whole brain; structural
and predicate selectors compose. Options: includeVectors, includeContent (VFS
blobs), includeSystem, edges ('induced'|'incident'|'none').
- import(backup, options?): onConflict 'merge' (dedup-by-id) | 'replace' | 'skip';
reembed 'auto' (re-embed from data when no vector carried) | 'never'; remapIds
to clone a subgraph under fresh ids.
- BackupData v1: format/formatVersion/brainyVersion/createdAt/embedding/entities/
relations/blobs?/danglingIds?/stats. Reserved fields (subtype, data, confidence,
weight, service) top-level; metadata custom-only. Current-state, no generations.
- Replaces the prior flat-entity export() (dropped relations) with a graph-complete
document. Distinct from brain.import(file) ingestion, which is unchanged.
- Export BackupData/BackupEntity/BackupRelation/ExportSelector/ExportOptions/
ImportOptions/ImportResult/DataAPI from the package root. CLI `brainy export`
now writes a BackupData document.
Guide: docs/guides/backup-and-export.md. Tests: tests/unit/api/data-backup.test.ts.
2026-06-16 15:52:42 -07:00
The format is versioned (`formatVersion` ) and current-state (no generation history) — see
the ** [Export & Import guide → ](../guides/export-and-import.md )**. This is distinct from
`brain.import(file)` (CSV/PDF/Excel/JSON ingestion).
2025-11-02 10:58:52 -08:00
---
## Configuration
### Constructor Options
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Storage configuration
storage: {
type: 'memory', // memory | opfs | filesystem | s3 | r2 | gcs | azure
path: './brainy-data', // For filesystem storage
compression: true, // Enable gzip compression (60-80% savings)
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Cloud storage configs (see Storage Adapters section)
s3Storage: { ... },
r2Storage: { ... },
gcsStorage: { ... },
azureStorage: { ... }
},
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// HNSW vector index config
hnsw: {
M: 16, // Connections per layer
efConstruction: 200, // Construction quality
efSearch: 100, // Search quality
typeAware: true // Enable type-aware indexing
},
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Model configuration (embedded in WASM - zero config needed)
// Model: all-MiniLM-L6-v2 (384 dimensions)
// Device: CPU via WASM (works everywhere)
2025-11-02 10:58:52 -08:00
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
// Cache configuration
cache: {
enabled: true,
maxSize: 10000,
ttl: 3600000 // 1 hour in ms
}
2025-11-02 10:58:52 -08:00
})
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
await brain.init() // Required! VFS auto-initialized
2025-11-02 10:58:52 -08:00
```
---
## Storage Adapters
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
All 7 storage adapters support **copy-on-write branching** .
2025-11-02 10:58:52 -08:00
### Memory (Default)
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: { type: 'memory' }
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Development, testing, prototyping
---
### OPFS (Browser)
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: { type: 'opfs' }
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Browser applications with persistent storage
---
### Filesystem (Node.js)
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: {
type: 'filesystem',
path: './brainy-data',
compression: true // 60-80% space savings
}
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Node.js applications, local persistence
---
### AWS S3
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: {
type: 's3',
s3Storage: {
bucketName: 'my-brainy-data',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
}
2025-11-02 10:58:52 -08:00
})
// Enable Intelligent-Tiering for 96% cost savings
await brain.storage.enableIntelligentTiering('entities/', 'auto-tier')
```
**Use case:** Production deployments, scalable storage
**[📖 AWS S3 Cost Optimization → ](../operations/cost-optimization-aws-s3.md )**
---
### Cloudflare R2
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: {
type: 'r2',
r2Storage: {
accountId: process.env.CF_ACCOUNT_ID,
bucketName: 'my-brainy-data',
accessKeyId: process.env.CF_ACCESS_KEY_ID,
secretAccessKey: process.env.CF_SECRET_ACCESS_KEY
}
}
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Zero egress fees, cost-effective storage
**[📖 R2 Cost Optimization → ](../operations/cost-optimization-cloudflare-r2.md )**
---
### Google Cloud Storage (GCS)
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: {
type: 'gcs',
gcsStorage: {
bucketName: 'my-brainy-data',
projectId: process.env.GCP_PROJECT_ID,
keyFilename: './gcp-key.json'
}
}
2025-11-02 10:58:52 -08:00
})
// Enable auto-tiering
await brain.storage.enableAutoclass({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
terminalStorageClass: 'ARCHIVE'
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Google Cloud ecosystem, global distribution
**[📖 GCS Cost Optimization → ](../operations/cost-optimization-gcs.md )**
---
### Azure Blob Storage
```typescript
const brain = new Brainy({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
storage: {
type: 'azure',
azureStorage: {
accountName: process.env.AZURE_STORAGE_ACCOUNT,
accountKey: process.env.AZURE_STORAGE_KEY,
containerName: 'brainy-data'
}
}
2025-11-02 10:58:52 -08:00
})
```
**Use case:** Azure ecosystem, enterprise deployments
**[📖 Azure Cost Optimization → ](../operations/cost-optimization-azure.md )**
---
## Utility Methods
### `clear()` → `Promise<void>`
Clear all data (entities and relationships).
```typescript
await brain.clear()
```
---
### `getNounCount()` → `Promise<number>`
Get total entity count.
```typescript
const count = await brain.getNounCount()
```
---
### `getVerbCount()` → `Promise<number>`
Get total relationship count.
```typescript
const count = await brain.getVerbCount()
```
---
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
### Subtype & facet APIs
Full guide: ** [Subtypes & Facets ](../guides/subtypes-and-facets.md )**.
#### `counts.bySubtype(type, subtype?)` → `Record<string, number> | number`
O(1) subtype counts for a NounType (backed by the persisted rollup).
```typescript
brain.counts.bySubtype(NounType.Person)
// → { employee: 12, customer: 847, vendor: 34 }
brain.counts.bySubtype(NounType.Person, 'employee')
// → 12
```
#### `counts.topSubtypes(type, n=10)` → `Array<[subtype, count]>`
Top N subtypes ranked by count.
```typescript
brain.counts.topSubtypes(NounType.Person, 3)
// → [['customer', 847], ['employee', 12], ['vendor', 34]]
```
#### `subtypesOf(type)` → `string[]`
Sorted distinct subtypes seen for a NounType.
```typescript
brain.subtypesOf(NounType.Person)
// → ['customer', 'employee', 'vendor']
```
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
#### `counts.byRelationshipSubtype(verb, subtype?)` → `Record<string, number> | number`
Verb-side mirror of `counts.bySubtype` . O(1) per-VerbType-per-subtype counts.
```typescript
brain.counts.byRelationshipSubtype(VerbType.ReportsTo)
// → { direct: 12, 'dotted-line': 3 }
brain.counts.byRelationshipSubtype(VerbType.ReportsTo, 'direct')
// → 12
```
#### `counts.topRelationshipSubtypes(verb, n=10)` → `Array<[subtype, count]>`
Top N subtypes for a `VerbType` ranked by count.
```typescript
brain.counts.topRelationshipSubtypes(VerbType.ReportsTo, 3)
// → [['direct', 12], ['dotted-line', 3]]
```
#### `relationshipSubtypesOf(verb)` → `string[]`
Sorted distinct subtypes seen for a `VerbType` .
```typescript
brain.relationshipSubtypesOf(VerbType.ReportsTo)
// → ['direct', 'dotted-line']
```
fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors
Brainy 7.30 shipped opt-in subtype enforcement; SDK 3.20.0 then registered
SDK_CORE_VOCABULARY on every consumer's brain (Event, Collection, Message,
Contract, Media, Document NounTypes). On 2026-06-08 Venue's /book flow went 500
because their brain.add({ type: NounType.Event, ... }) call sites lacked
subtype. An audit of Brainy's OWN source revealed 14 HIGH-risk internal write
paths that also omit subtype — any consumer running the same vocabulary would
have hit Brainy's infrastructure paths next. 7.30.1 closes both gaps before
8.0 makes strict mode the default.
Additive across the board. Zero behavior change for consumers not using strict
mode. Every change is JS-side — Cortex needs no work for 7.30.1.
NEW — brain.audit() diagnostic
- Read-only method walking storage.getNouns() / getVerbs() pagination
- Returns { entitiesWithoutSubtype: { type: count }, relationshipsWithoutSubtype,
total, scanned, recommendation }
- VFS infrastructure entities excluded by default (they bypass enforcement via
isVFSEntity marker); pass { includeVFS: true } to surface them
- The companion to migrateField (7.x) and fillSubtypes (8.0): tells consumers
exactly what would break under strict enforcement, deterministically
NEW — Improved enforcement error messages
- Caller's source location extracted from Error().stack so users see their own
call site, not a Brainy internal frame
- Specific guidance branches: registered vocabulary → "Pass one of: a, b, c";
brain-wide strict mode → mentions the except clause; otherwise → registration
recipe via brain.requireSubtype()
- Documentation link to the canonical migration recipe
- Same shape for noun and verb enforcement
NEW — CLI --subtype flag
- brainy add and brainy relate gain -s/--subtype <value>
- Defaults to 'cli-add' / 'cli-relate' so the CLI works against strict-mode
brains without the user needing to know the vocabulary in advance
INTERNAL — every Brainy write path now sets subtype
- VFS Contains edges (5 sites at lines 503/905/1694/1772/1886) → 'vfs-contains'
- VFS symlink entity → 'vfs-symlink' (NEW — distinct from 'vfs-file')
- VFS copy-file → preserves source subtype, falls back to 'vfs-file'
- VFS symlink also adopts the isVFSEntity infrastructure marker so it bypasses
enforcement in strict mode
- Aggregation materializer (Measurement entities) → 'materialized-aggregate'
- ImportCoordinator (3 sites): document → 'import-source'; entities →
options.defaultSubtype ?? 'imported'; placeholder → 'import-placeholder'
- SmartImportOrchestrator (4 entity sites + 2 batch relate sites): same
precedence (extractor → options.defaultSubtype → 'imported')
- EntityDeduplicator → candidate.subtype ?? 'imported'
- UniversalImportAPI → extractor → 'extracted' for both entities and relations
- NeuralImport → adds defaultSubtype to NeuralImportOptions; precedence same
- GoogleSheetsIntegration → request body 'subtype' ?? 'imported-from-sheets'
- ODataIntegration → request body 'Subtype' ?? 'imported-from-odata'
- MCP client message storage → 'mcp-message' (also fixes pre-existing missing
data field and missing type by aliasing from the prior text field)
Side-effect fix: storage.getNouns() paginated now surfaces subtype to top-level
- Single-noun getNoun() already did this in 7.30; the paginated path was missed
- Without this fix brain.audit() saw missing subtype on entities that actually
had one (caught by the strict-mode self-test before release)
NEW — tests/integration/strict-mode-self-test.test.ts (13 tests)
- Creates a brain under the exact SDK_CORE_VOCABULARY shape Venue hit + brain-
wide strict mode
- Exercises every internal Brainy path: VFS root + mkdir + writeFile + cp + mv
+ ln + symlink; aggregation engine; audit diagnostic with includeVFS toggle
- Validates error message UX: caller location, vocabulary guidance, brain-wide
strict mode guidance, off-vocabulary value reporting
Docs
- New "Strict mode in practice" section in docs/guides/subtypes-and-facets.md
covering the SDK_CORE_VOCABULARY pattern, 4-step migration recipe
(audit → migrateField → hand-fix → re-audit), the Brainy-internal label
reference table, and an 8.0 forward-look on fillSubtypes()
- docs/api/README.md: new audit() entry, strict-mode tips on add() and relate()
- RELEASES.md: full 7.30.1 entry
Cortex parity (forward-looking, not blocking 7.30.1)
- 6th open question added to .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md: native
fast path for audit() and fillSubtypes() via column-store null-subtype
bitmap for billion-scale brains
- Cortex should add a parity test mirroring strict-mode-self-test.test.ts
against their native paths to catch any latent bug where native writes
bypass JS validation
- Brainy-internal subtype labels become a documented part of the 8.0 contract
(useful for Cortex telemetry surfacing Brainy-managed infrastructure %)
Verification
- npx tsc --noEmit: clean
- npm test: 1468/1468 unit
- 7.29 noun integration suite: 26/26 (no regression)
- 7.30 verb subtype + enforcement integration suite: 30/30 (no regression)
- New strict-mode-self-test integration suite: 13/13
- npm run build: clean
- Closed-source product reference audit: clean
Addresses VE-SUBTYPE-MIGRATION (Venue's reported request) and ships internal
labels Venue did NOT ask for but that would have broken them next under their
own vocabulary registration.
2026-06-08 11:31:47 -07:00
#### `audit(options?)` → `Promise<AuditReport>` (7.30.1+)
Diagnostic — find entities and relationships missing a `subtype` value, grouped by type. The companion to `migrateField()` / `fillSubtypes()` — answers "what would break if I enabled strict subtype enforcement?".
```typescript
const report = await brain.audit()
// {
// entitiesWithoutSubtype: { event: 24, document: 3 },
// relationshipsWithoutSubtype: { relatedTo: 1402 },
// total: 1429,
// scanned: 8400,
// recommendation: 'Found 1429 entries without subtype. ...'
// }
```
**Parameters:**
- `options.includeVFS?` : `boolean` — When `false` (default), VFS infrastructure entities (`metadata.isVFSEntity` / `metadata.isVFS` ) are excluded. They bypass enforcement anyway, so counting them is noise.
- `options.batchSize?` : `number` — Pagination batch size (default 200).
- `options.onProgress?` : `(progress: { scanned, missingSubtype }) => void` — Progress callback per batch.
Run before adopting an SDK that registers `requireSubtype()` rules, or before upgrading to Brainy 8.0 (which makes strict mode the default). See the [Strict mode in practice ](../guides/subtypes-and-facets.md#strict-mode-in-practice-for-sdk-style-vocabulary-consumers ) guide for the full migration recipe.
feat: verb subtype + updateRelation + requireSubtype enforcement
Brings verbs to first-class parity with nouns. The 7.29.0 subtype primitive
shipped for entities only; this release ships the symmetric verb mirror plus
the enforcement layer for ensuring every entity AND every relationship has
both type AND subtype.
Layer V1 — verb subtype mirror
- HNSWVerbWithMetadata.subtype + STANDARD_VERB_FIELDS set + resolveVerbField()
- Relation<T>.subtype, RelateParams<T>.subtype, UpdateRelationParams<T> extended,
GetRelationsParams.subtype, GraphConstraints.subtype (for find connected)
- relate() persists subtype on verbMetadata + GraphVerb + transaction ops
- getRelations({ type, subtype }) fast-path filter with set membership
- find({ connected: { via, subtype, depth } }) traversal filter (depth-1 on
the JS path; explicit error on depth > 1 pointing at Cortex native)
- verbsToRelations + storage destructure sites surface subtype to top-level
- All three graph-index fast-path queries (getVerbsBySource/ByTarget) enrich
with subtype from metadata
Layer V2 — updateRelation() closes a pre-7.30 gap
- New first-class verb update method (parallel to update() for nouns)
- Changes subtype/type/weight/confidence/data/metadata in place
- Re-indexes in graph adjacency when verb type changes; id preserved
- validateUpdateRelationParams enforces id + at-least-one-field-to-update
Layer V3 — verb subtype storage rollup
- verbSubtypeCountsByType: Map<number, Map<string, number>> on BaseStorage
- verbSubtypeByIdCache for self-heal during update/delete
- incrementVerbSubtypeCount + decrementVerbSubtypeCount maintain state
- loadVerbSubtypeStatistics + saveVerbSubtypeStatistics persist to
_system/verb-subtype-statistics.json (mirrors noun-side shape)
- rebuildVerbSubtypeCounts for poison recovery / explicit repair
- getVerbSubtypeCountsByType accessor for the public counts API
- Wired into init() / flushCounts() / saveVerbMetadata / deleteVerbMetadata
Layer V4 — verb counts API + relationshipSubtypesOf
- brain.counts.byRelationshipSubtype(verb, subtype?) — O(1) breakdown or point
- brain.counts.topRelationshipSubtypes(verb, n) — top N by count
- brain.relationshipSubtypesOf(verb) — sorted distinct subtypes
Layer V5 — migrateField extended to verbs
- New entityKind?: 'noun' | 'verb' | 'both' option (default 'noun')
- Mirror verb iteration via storage.getVerbs() with same path semantics
- verbToRelationLike + buildRelationMigrationUpdate helpers project the
storage verb shape onto the Entity<T>-shaped surface readPath understands
- Routes through new updateRelation() for the verb-side rewrite
Enforcement (opt-in in 7.30, default in 8.0)
- brain.requireSubtype(type, options) — unified API for NounType OR VerbType.
Registers per-type rules with optional values whitelist; composes with the
brain-wide flag.
- new Brainy({ requireSubtype: true }) — brain-wide strict mode. Every public
write path validates the pairing guarantee.
- { except: [NounType.Thing, ...] } form for catch-all type exemptions
- Atomic-fail semantics on addMany / relateMany — pre-validate every item
before any storage write, throw on first failure with item index
- Per-type rules + brain-wide flag both throw with descriptive messages
- VFS infrastructure bypass via metadata.isVFSEntity / isVFS markers so
brain's own VFS writes don't get rejected when strict mode is on
VFS labeling — concrete subtypes for infrastructure entities
- VFS root: NounType.Collection + subtype: 'vfs-root' (was bare Collection)
- VFS directories: subtype: 'vfs-directory'
- VFS files: subtype: 'vfs-file' (NounType still mime-based)
- VFS containment edges: VerbType.Contains + subtype: 'vfs-contains'
- Lets consumers cleanly enumerate VFS state via find({ subtype: 'vfs-file' })
and distinguish Brainy's VFS Collections from user-created Collections
Docs
- docs/guides/subtypes-and-facets.md extended with Layer V (Verbs) section +
Enforcement section. New full reference at the bottom split into Layer 1
(nouns), Layer V (verbs), Layer 2 (facets), Layer 3 (migration), Enforcement.
- docs/api/README.md adds updateRelation(), getRelations({ subtype }), the
three verb-side counts methods, requireSubtype(), and the brain-wide
constructor option. relate() params include subtype.
- docs/DATA_MODEL.md adds a Subtype-for-VerbType section + STANDARD_VERB_FIELDS
- docs/architecture/finite-type-system.md extends Principle 1a to verbs
- docs/QUERY_OPERATORS.md adds a verb-subtype filter section covering
getRelations and find({connected, subtype}) traversal
- README.md "Subtypes" section now shows both noun + verb in one example +
the enforcement APIs
- RELEASES.md v7.30.0 entry with the full noun/verb capability parity matrix
Tests
- tests/integration/verb-subtype-and-enforcement.test.ts — 30 new tests
covering V1 round-trips, V1 set membership, updateRelation in place,
updateRelation preservation, V2 counts breakdown + point + topN + distinct,
V2 decrements on unrelate, V2 re-routes on updateRelation, V3 depth-1
traversal filter, V3 depth>1 explicit error, V4 verb migration, V4 both
entity kinds, V4 readBoth preservation, V5 per-type required rejection,
V5 vocabulary rejection, V5 on-vocab acceptance, V5 verb-side enforcement,
V5 addMany atomic-fail, V5 relateMany atomic-fail, V5 update enforcement,
V5 updateRelation enforcement, V5 brain-wide strict mode, V5 except clause.
Verification
- Unit suite: 1468/1468 passing
- Noun subtype integration (7.29 carryover): 26/26 passing
- Verb subtype + enforcement integration: 30/30 passing
- Type-check: clean
- Build: clean
- Public closed-source reference audit: clean
Internal 8.0 spec
- .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md (gitignored, not in npm artifact)
documents the contract upgrade Cortex 3.0 implements against: required-by-
default subtype, SubtypeRegistry typing hook, native simplification,
multi-hop traversal native fast path, brain.fillSubtypes() migration helper.
Coordinated via PLATFORM-HANDOFF rows CTX-SUBTYPE-PARITY-V2 (7.30 parallel
work) and CTX-SUBTYPE-8.0-CONTRACT (8.0 spec).
2026-06-05 11:15:52 -07:00
#### `requireSubtype(type, options?)` → `void`
Register subtype enforcement for a specific `NounType` or `VerbType` . Unified API for nouns and verbs. Composes with the brain-wide `requireSubtype` constructor flag.
```typescript
// Lock down Person sub-classification
brain.requireSubtype(NounType.Person, {
values: ['employee', 'customer', 'vendor'],
required: true
})
// Lock down management edges
brain.requireSubtype(VerbType.ReportsTo, {
values: ['direct', 'dotted-line'],
required: true
})
```
**Parameters:**
- `type` : `NounType | VerbType` - The type to register
- `options.values?` : `string[]` - Vocabulary whitelist (rejects off-vocab values)
- `options.required?` : `boolean` - Whether subtype is required (default: `true` )
#### Brain-wide strict mode — `new Brainy({ requireSubtype })`
Constructor option that enforces subtype on every `add()` / `addMany()` / `update()` / `relate()` / `relateMany()` / `updateRelation()` for every type:
```typescript
// Every write must include subtype
const brain = new Brainy({ requireSubtype: true })
// Exempt specific types (e.g. catch-all Thing)
const brain2 = new Brainy({
requireSubtype: { except: [NounType.Thing, NounType.Custom] }
})
```
When strict mode is on:
- Every public write path checks the pairing guarantee.
- `addMany()` / `relateMany()` validate all items BEFORE any storage write — atomic-fail, no partial writes.
- Brainy's own VFS infrastructure writes bypass via the `metadata.isVFSEntity: true` marker.
- Per-type registrations always apply regardless of the brain-wide flag.
Becomes the default in 8.0.0.
feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).
Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
BaseStorage, mirrored after nounCountsByType with the same self-heal
rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path
Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
cardinality + per-NounType breakdown stats. Backed by the aggregation
engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update
Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
streams every entity, copies the value from one path to another, and
(unless readBoth) clears the source. Supports top-level standard fields,
metadata.X, and data.X paths. Idempotent — safe to re-run.
Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
quick-start all treat subtype as a core primitive with anonymous example
vocabularies (employee/customer/invoice/milestone).
Tests
- 26 new integration tests covering write/read/update/delete round-trips,
counts rollup decrement + re-route on mutation, trackField + byField
with and without perType, vocabulary whitelist enforcement, and
migrateField for metadata.X → subtype and data.X → subtype paths
including readBoth deprecation-window semantics.
Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
#### `trackField(name, options?)` → `void`
Register a metadata field for cardinality + per-NounType breakdown stats. With `values: [...]` , validates against the whitelist on `add()` /`update()` .
```typescript
brain.trackField('status') // basic
brain.trackField('status', { perType: true }) // with per-NounType breakdown
brain.trackField('priority', { values: ['low', 'med', 'high'] }) // strict vocabulary
```
#### `counts.byField(name, options?)` → `Promise<Record<string, number>>`
Counts by value for a tracked field. Requires `perType: true` registration if filtering by NounType.
```typescript
await brain.counts.byField('status')
// → { todo: 12, doing: 3, done: 47 }
await brain.counts.byField('status', { type: NounType.Task })
// → { todo: 8, doing: 2, done: 30 }
```
#### `migrateField(options)` → `Promise<MigrationSummary>`
Stream-and-rewrite a field across the brain. Supports `metadata.X` , `data.X` , and top-level paths. Idempotent.
```typescript
// One-shot rewrite
await brain.migrateField({ from: 'metadata.kind', to: 'subtype' })
// Deprecation window — keep source field readable
await brain.migrateField({ from: 'data.kind', to: 'subtype', readBoth: true })
// With progress reporting
await brain.migrateField({
from: 'metadata.kind',
to: 'subtype',
batchSize: 500,
onProgress: ({ scanned, migrated }) => console.log(`${scanned} / ${migrated}` )
})
```
Returns `{ scanned: number, migrated: number, skipped: number, errors: Array<{id, error}> }` .
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `embed(data)` → `Promise<number[]>` ✨
2025-11-02 10:58:52 -08:00
2026-01-06 14:52:12 -08:00
Generate embedding vector from text or data.
2025-11-02 10:58:52 -08:00
```typescript
const vector = await brain.embed('Hello world')
2026-01-06 14:52:12 -08:00
// 384-dimensional vector
console.log(vector.length) // 384
2025-11-02 10:58:52 -08:00
```
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `embedBatch(texts)` → `Promise<number[][]>` ✨
2026-01-06 14:52:12 -08:00
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
Batch embed multiple texts using native WASM batch API (single forward pass).
2026-01-06 14:52:12 -08:00
```typescript
const embeddings = await brain.embedBatch([
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
'Machine learning is fascinating',
'Deep neural networks',
'Natural language processing'
2026-01-06 14:52:12 -08:00
])
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(embeddings.length) // 3
2026-01-06 14:52:12 -08:00
console.log(embeddings[0].length) // 384
```
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
> Uses the WASM engine's native `embed_batch()` for a single model forward pass instead of N individual calls. This is the same batch API used internally by `highlight()`.
feat: add structured content extraction and batch embedding optimization to highlight()
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
2026-01-06 14:52:12 -08:00
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `similarity(textA, textB)` → `Promise<number>` ✨
2026-01-06 14:52:12 -08:00
Calculate semantic similarity between two texts.
```typescript
const score = await brain.similarity(
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
'The cat sat on the mat',
'A feline was resting on the rug'
2026-01-06 14:52:12 -08:00
)
console.log(score) // ~0.85 (high semantic similarity)
```
**Returns:** Score from 0 (different) to 1 (identical meaning)
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `neighbors(entityId, options?)` → `Promise<string[]>` ✨
2026-01-06 14:52:12 -08:00
Get graph neighbors of an entity.
```typescript
// Get all connected entities
const neighbors = await brain.neighbors(entityId)
// Get outgoing connections only
const outgoing = await brain.neighbors(entityId, {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
direction: 'outgoing',
limit: 10
2026-01-06 14:52:12 -08:00
})
// Multi-hop traversal
const extended = await brain.neighbors(entityId, {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
depth: 2,
direction: 'both'
2026-01-06 14:52:12 -08:00
})
```
**Options:**
- `direction` : `'outgoing' | 'incoming' | 'both'` (default: 'both')
- `depth` : `number` - Traversal depth (default: 1)
- `verbType` : `VerbType` - Filter by relationship type
- `limit` : `number` - Maximum neighbors to return
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `findDuplicates(options?)` → `Promise<DuplicateResult[]>` ✨
2026-01-06 14:52:12 -08:00
Find semantic duplicates in the database.
```typescript
// Find all duplicates
const duplicates = await brain.findDuplicates()
for (const group of duplicates) {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log('Original:', group.entity.id)
for (const dup of group.duplicates) {
console.log(` Duplicate: ${dup.entity.id} (${dup.similarity.toFixed(2)})` )
}
2026-01-06 14:52:12 -08:00
}
// Find person duplicates with higher threshold
const personDupes = await brain.findDuplicates({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
type: NounType.PERSON,
threshold: 0.9,
limit: 50
2026-01-06 14:52:12 -08:00
})
```
**Options:**
- `threshold` : `number` - Minimum similarity (default: 0.85)
- `type` : `NounType` - Filter by entity type
- `limit` : `number` - Maximum duplicate groups (default: 100)
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `indexStats()` → `Promise<IndexStats>` ✨
2026-01-06 14:52:12 -08:00
Get comprehensive index statistics.
```typescript
const stats = await brain.indexStats()
console.log(`Entities: ${stats.entities}` )
console.log(`Vectors: ${stats.vectors}` )
console.log(`Relationships: ${stats.relationships}` )
console.log(`Memory: ${(stats.memoryUsage.total / 1024 / 1024).toFixed(1)}MB` )
console.log(`Fields: ${stats.metadataFields.join(', ')}` )
```
**Returns:**
- `entities` - Total entity count
- `vectors` - Total vectors in HNSW index
- `relationships` - Total relationships in graph
- `metadataFields` - Indexed metadata fields
- `memoryUsage.vectors` - Vector memory (bytes)
- `memoryUsage.graph` - Graph memory (bytes)
- `memoryUsage.metadata` - Metadata index memory (bytes)
- `memoryUsage.total` - Total memory usage
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### `cluster(options?)` → `Promise<ClusterResult[]>` ✨
2026-01-06 14:52:12 -08:00
Cluster entities by semantic similarity.
```typescript
// Find all clusters
const clusters = await brain.cluster()
for (const cluster of clusters) {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
console.log(`${cluster.clusterId}: ${cluster.entities.length} entities` )
2026-01-06 14:52:12 -08:00
}
// Find document clusters with centroids
const docClusters = await brain.cluster({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
type: NounType.Document,
threshold: 0.85,
minClusterSize: 3,
includeCentroid: true
2026-01-06 14:52:12 -08:00
})
```
**Options:**
- `threshold` : `number` - Similarity threshold (default: 0.8)
- `type` : `NounType` - Filter by entity type
- `minClusterSize` : `number` - Minimum cluster size (default: 2)
- `limit` : `number` - Maximum clusters to return (default: 100)
- `includeCentroid` : `boolean` - Calculate cluster centroids (default: false)
**Returns:**
- `clusterId` - Unique cluster identifier
- `entities` - Array of entities in the cluster
- `centroid` - Average embedding vector (if includeCentroid is true)
---
2025-11-02 10:58:52 -08:00
### `getStats()` → `Statistics`
Get comprehensive statistics.
```typescript
const stats = brain.getStats()
console.log(stats.entityCount)
console.log(stats.relationshipCount)
console.log(stats.cacheHitRate)
```
---
## Lifecycle
### Initialization
```typescript
const brain = new Brainy(config)
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
await brain.init() // Required! VFS auto-initialized here
2025-11-02 10:58:52 -08:00
```
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
VFS is auto-initialized during `brain.init()` - no separate `vfs.init()` needed!
2025-11-02 10:58:52 -08:00
---
### Shutdown
```typescript
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
await brain.shutdown() // Graceful shutdown, flush caches
2025-11-02 10:58:52 -08:00
```
---
## Examples
### Basic CRUD
```typescript
// Create
const id = await brain.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'Quantum computing breakthrough',
type: NounType.Concept,
metadata: { category: 'tech', year: 2024 }
2025-11-02 10:58:52 -08:00
})
// Read
const entity = await brain.get(id)
// Update
await brain.update({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
id,
metadata: { updated: true }
2025-11-02 10:58:52 -08:00
})
// Delete
await brain.delete(id)
```
---
### Knowledge Graphs
```typescript
// Create entities
const ai = await brain.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'Artificial Intelligence',
type: NounType.Concept
2025-11-02 10:58:52 -08:00
})
const ml = await brain.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'Machine Learning',
type: NounType.Concept
2025-11-02 10:58:52 -08:00
})
// Create relationship
await brain.relate({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
from: ml,
to: ai,
type: VerbType.IsA
2025-11-02 10:58:52 -08:00
})
// Traverse graph
const results = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
connected: { from: ai, depth: 2 }
2025-11-02 10:58:52 -08:00
})
```
---
### Triple Intelligence Query
```typescript
const results = await brain.find({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
query: 'modern frontend frameworks', // 🔍 Vector
where: { // 📊 Document
year: { greaterThan: 2020 },
category: { oneOf: ['framework', 'library'] }
},
connected: { // 🕸️ Graph
to: reactId,
depth: 2,
type: VerbType.BuiltOn
},
limit: 10
2025-11-02 10:58:52 -08:00
})
```
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### Git-Style Workflow
2025-11-02 10:58:52 -08:00
```typescript
// Fork for experimentation
const experiment = await brain.fork('test-migration')
// Make changes in isolation
await experiment.add({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
data: 'New feature',
type: NounType.Document
2025-11-02 10:58:52 -08:00
})
// Commit your work
await experiment.commit({
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
message: 'Add new feature',
author: 'dev@example .com'
2025-11-02 10:58:52 -08:00
})
feat: ID-first storage architecture + remove memory-unsafe APIs (v6.0.0)
BREAKING CHANGES:
**ID-First Storage Paths**
- Direct O(1) entity access without type lookups
- Before: entities/nouns/{TYPE}/metadata/{SHARD}/{ID}.json
- After: entities/nouns/{SHARD}/{ID}/metadata.json
- Migration handled automatically on first init()
**Removed Memory-Unsafe APIs**
- Removed brain.merge() - loaded all entities into memory
- Removed brain.diff() - loaded all entities into memory
- Removed brain.data().backup() - loaded all entities into memory
- Removed brain.data().restore() - depended on backup()
- Removed CLI commands: backup, restore, cow merge
**Migration Paths**
- merge() → Use checkout() or manually copy entities with pagination
- diff() → Use asOf() with manual paginated comparison
- backup() → Use fork() for instant COW snapshots
- restore() → Use checkout() to switch to snapshot branch
Core Improvements:
- ✅ All 8 storage adapters properly call super.init()
- ✅ GraphAdjacencyIndex integration in BaseStorage.init()
- ✅ Fixed ID-first path bugs (vector.json → vectors.json)
- ✅ Fixed MemoryStorage.initializeCounts() for ID-first paths
- ✅ New VFS APIs: du(), access(), find()
- ✅ Comprehensive documentation with migration guides
Storage Adapters Fixed:
- MemoryStorage, FileSystemStorage, AzureBlobStorage
- GCSStorage, R2Storage, S3CompatibleStorage
- OPFSStorage, HistoricalStorageAdapter
Files Changed: 28 files, +1,075/-1,933 lines (net -858)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 16:46:11 -08:00
// Switch to experimental branch to make it active
await brain.checkout('test-migration')
2025-11-02 10:58:52 -08:00
```
---
### VFS File Management
```typescript
// Write files
await brain.vfs.writeFile('/docs/README.md', 'Project documentation')
await brain.vfs.mkdir('/src/components', { recursive: true })
// Read files
const content = await brain.vfs.readFile('/docs/README.md')
// Semantic search
const reactFiles = await brain.vfs.search('React components with hooks', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
path: '/src'
2025-11-02 10:58:52 -08:00
})
// Get tree structure (safe, prevents infinite recursion)
const tree = await brain.vfs.getTreeStructure('/projects', {
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
maxDepth: 3
2025-11-02 10:58:52 -08:00
})
```
---
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
## Type System Reference
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
Stage 3 CANONICAL taxonomy with 169 types (42 nouns + 127 verbs)
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
### Noun Types (42)
Brainy uses a comprehensive noun type system covering 96-97% of human knowledge:
**Core Entity Types (7)**
- `NounType.Person` - Individual human entities
- `NounType.Organization` - Companies, institutions, collectives
- `NounType.Location` - Geographic and spatial entities
- `NounType.Thing` - Physical objects and artifacts
- `NounType.Concept` - Abstract ideas and principles
- `NounType.Event` - Temporal occurrences
- `NounType.Agent` - AI agents, bots, automated systems
**Digital/Content Types (4)**
- `NounType.Document` - Text-based files and written content
- `NounType.Media` - Audio, video, images
- `NounType.File` - Generic digital files
- `NounType.Message` - Communication content
**Business Types (4)**
- `NounType.Product` - Commercial products
- `NounType.Service` - Service offerings
- `NounType.Task` - Actions, todos, work items
- `NounType.Project` - Organized initiatives
**Scientific Types (2)**
- `NounType.Hypothesis` - Theories and propositions
- `NounType.Experiment` - Studies and investigations
**And 25 more types** including: `Organism` , `Substance` , `Quality` , `TimeInterval` , `Function` , `Proposition` , `Collection` , `Dataset` , `Process` , `State` , `Role` , `Language` , `Currency` , `Measurement` , `Contract` , `Regulation` , `Interface` , `Resource` , `Custom` , `SocialGroup` , `Institution` , `Norm` , `InformationContent` , `InformationBearer` , `Relationship`
### Verb Types (127)
Brainy supports 127 relationship types organized into categories:
**Foundational (7)**
- `VerbType.InstanceOf` , `VerbType.SubclassOf` , `VerbType.ParticipatesIn`
- `VerbType.RelatedTo` , `VerbType.Contains` , `VerbType.PartOf` , `VerbType.References`
**Spatial & Temporal (14)**
- Location: `LocatedAt` , `AdjacentTo` , `ContainsSpatially` , `OverlapsSpatially` , `Above` , `Below` , `Inside` , `Outside` , `Facing`
- Time: `Precedes` , `During` , `OccursAt` , `Overlaps` , `ImmediatelyAfter` , `SimultaneousWith`
**Causal & Dependency (11)**
- Direct: `Causes` , `Enables` , `Prevents` , `DependsOn` , `Requires`
- Modal: `CanCause` , `MustCause` , `WouldCauseIf` , `ProbablyCauses`
- Variations: `RigidlyDependsOn` , `FunctionallyDependsOn` , `HistoricallyDependsOn`
**Creation & Change (10)**
- Lifecycle: `Creates` , `Transforms` , `Becomes` , `Modifies` , `Consumes` , `Destroys`
- Properties: `GainsProperty` , `LosesProperty` , `RemainsSame` , `PersistsThrough`
**Social & Communication (8)**
- `MemberOf` , `WorksWith` , `FriendOf` , `Follows` , `Likes` , `ReportsTo` , `Mentors` , `Communicates`
**Epistemic & Modal (14)**
- Knowledge: `Knows` , `Doubts` , `Believes` , `Learns`
- Mental states: `Desires` , `Intends` , `Fears` , `Loves` , `Hates` , `Hopes` , `Perceives`
- Modality: `CouldBe` , `MustBe` , `Counterfactual`
**Measurement & Comparison (9)**
- `Measures` , `MeasuredIn` , `ConvertsTo` , `HasMagnitude` , `GreaterThan`
- `SimilarityDegree` , `ApproximatelyEquals` , `MoreXThan` , `HasDegree`
**And 54 more specialized verbs** including ownership, composition, uncertainty, deontic relationships (obligations/permissions), context-dependent truth, spatial/temporal variations, information theory, and meta-level relationships.
### Complete Reference
For the full taxonomy with all 169 types and their descriptions, see:
- **[Stage 3 CANONICAL Taxonomy ](../STAGE3-CANONICAL-TAXONOMY.md )** - Complete list with categories
- **[Noun-Verb Taxonomy Architecture ](../architecture/noun-verb-taxonomy.md )** - Design rationale
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
### Migration from
docs: complete Stage 3 CANONICAL taxonomy documentation (v5.5.0)
Added comprehensive documentation for Stage 3 taxonomy migration:
API Documentation Updates:
- Updated version header from v5.4.0 to v5.5.0
- Fixed all deprecated NounType.Content examples → Document, Concept
- Added versions.asOf() method documentation with examples
- Added comprehensive Type System Reference section
Type System Reference:
- Documented all 42 noun types organized by category
- Documented 127 verb types organized by functional groups
- Added migration guide from v5.4.0 (deprecated types)
- Breaking changes: User→Person, Topic→Concept, Content→Document
- Stage 3 adds +11 nouns, +87 verbs (169 total types)
Build Status:
- ✅ TypeScript build passes with zero errors (stale cache issue resolved)
- ✅ All Stage 2 references updated to Stage 3 CANONICAL
- ✅ Type embeddings current (42 nouns + 127 verbs = 338KB)
Tested with confidential Tales from Talifar Glossary (NOT in repo):
- Successfully imported 2,284 entities across 7 noun types
- Created 2,280 relationships (contains verb)
- Noun type distribution: document (50%), thing (16%), person (16%),
location (9%), concept (9%), collection (1%), file (<1%)
- Demonstrates Stage 3 taxonomy handles rich fantasy world data
Type System Coverage:
- 7/42 noun types used (16.7%) - appropriate for glossary domain
- 1/127 verb types used (0.8%) - opportunity for richer relationships
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 10:28:47 -08:00
**Breaking Changes:**
- `NounType.Content` removed → Use `Document` , `Message` , or `InformationContent`
- `NounType.User` removed → Use `Person` or `Agent`
- `NounType.Topic` removed → Use `Concept` or `Category`
**New Types Added:**
- **+11 noun types**: Agent, Organism, Substance, Quality, TimeInterval, Function, Proposition, Custom, SocialGroup, Institution, Norm, InformationContent, InformationBearer, Relationship
- **+87 verb types**: Extensive additions across all categories
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
## Key Features
2025-11-04 11:19:02 -08:00
- ✅ **Entity Versioning** - Git-style versioning for individual entities
- ✅ **Content-Addressable Storage** - SHA-256 deduplication for versions
- ✅ **Auto-Versioning Augmentation** - Automatic version creation on updates
- ✅ **Branch-Isolated Versions** - Versions isolated per branch
- ✅ **VFS Entity Filtering** - All VFS entities now have `isVFSEntity: true` flag
2025-11-02 10:58:52 -08:00
- ✅ **VFS Auto-Initialization** - No more separate `vfs.init()` calls
- ✅ **VFS Property Access** - Use `brain.vfs.method()` instead of `brain.vfs().method()`
- ✅ **Complete COW Support** - All 20 TypeAware methods use COW helpers
- ✅ **Verified Import/Export** - Work correctly with current branch
- ✅ **Instant Fork** - Snowflake-style copy-on-write (< 100ms fork time )
feat: ID-first storage architecture + remove memory-unsafe APIs (v6.0.0)
BREAKING CHANGES:
**ID-First Storage Paths**
- Direct O(1) entity access without type lookups
- Before: entities/nouns/{TYPE}/metadata/{SHARD}/{ID}.json
- After: entities/nouns/{SHARD}/{ID}/metadata.json
- Migration handled automatically on first init()
**Removed Memory-Unsafe APIs**
- Removed brain.merge() - loaded all entities into memory
- Removed brain.diff() - loaded all entities into memory
- Removed brain.data().backup() - loaded all entities into memory
- Removed brain.data().restore() - depended on backup()
- Removed CLI commands: backup, restore, cow merge
**Migration Paths**
- merge() → Use checkout() or manually copy entities with pagination
- diff() → Use asOf() with manual paginated comparison
- backup() → Use fork() for instant COW snapshots
- restore() → Use checkout() to switch to snapshot branch
Core Improvements:
- ✅ All 8 storage adapters properly call super.init()
- ✅ GraphAdjacencyIndex integration in BaseStorage.init()
- ✅ Fixed ID-first path bugs (vector.json → vectors.json)
- ✅ Fixed MemoryStorage.initializeCounts() for ID-first paths
- ✅ New VFS APIs: du(), access(), find()
- ✅ Comprehensive documentation with migration guides
Storage Adapters Fixed:
- MemoryStorage, FileSystemStorage, AzureBlobStorage
- GCSStorage, R2Storage, S3CompatibleStorage
- OPFSStorage, HistoricalStorageAdapter
Files Changed: 28 files, +1,075/-1,933 lines (net -858)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 16:46:11 -08:00
- ✅ **Git-Style Branching** - fork, commit, checkout, listBranches
2025-11-02 10:58:52 -08:00
- ✅ **Full Branch Isolation** - Parent and fork fully isolated
- ✅ **Read-Through Inheritance** - Forks see parent + own data
- ✅ **Universal Storage Support** - All 7 adapters support branching
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
**[📖 Complete Changes → ](../../.strategy/v5.1.0-CHANGES.md )**
2025-11-02 10:58:52 -08:00
---
## Support & Resources
- **📖 Documentation:** [Full Documentation ](../ )
- **🐛 Issues:** [GitHub Issues ](https://github.com/soulcraftlabs/brainy/issues )
- **💬 Discussions:** [GitHub Discussions ](https://github.com/soulcraftlabs/brainy/discussions )
- **📦 NPM:** [@soulcraft/brainy ](https://www.npmjs.com/package/@soulcraft/brainy )
- **⭐ GitHub:** [Star us ](https://github.com/soulcraftlabs/brainy )
---
## See Also
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- **[Data Model ](../DATA_MODEL.md )** - Entity structure, data vs metadata, storage fields
- **[Query Operators ](../QUERY_OPERATORS.md )** - All BFO operators with examples and indexed vs in-memory matrix
2025-11-02 10:58:52 -08:00
- **[Triple Intelligence Architecture ](../architecture/triple-intelligence.md )** - How vector + graph + document work together
feat: enforce data/metadata separation, numeric range queries, improved docs
- Store data opaquely in add() and update() instead of spreading object
properties into top-level metadata. data is for semantic search (HNSW),
metadata is for structured where-filter queries (MetadataIndex).
- Fix numeric range queries in MetadataIndex — use numeric-aware comparison
instead of lexicographic string comparison for normalized values.
- Add data field to RelateParams and Relation types for relationship content.
- Add where.type → where.noun alias in metadata-only find() path.
- Rewrite README: focused ~350 lines from 791, quick start first, feature
showcase with mini-snippets, organized doc links, no version callouts.
- Add DATA_MODEL.md and QUERY_OPERATORS.md reference docs.
- Remove 10 outdated/redundant doc files consolidated into API reference.
- Improve JSDoc on Entity, Relation, AddParams, FindParams, and core methods.
- Fix tests asserting data properties appear in metadata (data model violation).
- Deprecate verb.source/target in favor of from/to (public) and sourceId/targetId (storage).
2026-02-09 12:06:59 -08:00
- **[Find System ](../FIND_SYSTEM.md )** - Natural language find() details
2025-11-02 10:58:52 -08:00
- **[VFS Quick Start ](../vfs/QUICK_START.md )** - Complete VFS documentation
- **[Import Anything Guide ](../guides/import-anything.md )** - CSV, Excel, PDF, URL imports
- **[Cloud Deployment ](../deployment/CLOUD_DEPLOYMENT_GUIDE.md )** - Production deployment
- **[Instant Fork ](../features/instant-fork.md )** - Git-style branching guide
---
2025-08-26 12:32:21 -07:00
2025-11-02 10:58:52 -08:00
**License:** MIT © Brainy Contributors
2025-08-26 12:32:21 -07:00
---
feat: expand ContentCategory to universal 6-category set for highlight()
Replace document-centric categories (prose/heading/code/label) with a
universal set that works across documents, code, and UI:
- title: headings, identifiers, labels, JSON keys
- annotation: comments, docstrings, captions, alt text
- content: paragraphs, list items, flowing text
- value: string literals, numbers, form values
- code: unparsed code blocks
- structural: keywords, operators, punctuation
Built-in extractors now produce title/content/code. All 6 categories
are available for custom parsers (e.g. tree-sitter).
Also adds inline code detection in Markdown: backtick spans within
prose lines are split into separate code/content segments.
2026-01-27 13:44:58 -08:00
*Brainy - The Knowledge Operating System*
2025-11-02 10:58:52 -08:00
*From prototype to planet-scale • Zero configuration • Triple Intelligence™ • Git-Style Branching*