brainy/docs
David Snelling 5f3a2ca7d5 fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors
Brainy 7.30 shipped opt-in subtype enforcement; SDK 3.20.0 then registered
SDK_CORE_VOCABULARY on every consumer's brain (Event, Collection, Message,
Contract, Media, Document NounTypes). On 2026-06-08 Venue's /book flow went 500
because their brain.add({ type: NounType.Event, ... }) call sites lacked
subtype. An audit of Brainy's OWN source revealed 14 HIGH-risk internal write
paths that also omit subtype — any consumer running the same vocabulary would
have hit Brainy's infrastructure paths next. 7.30.1 closes both gaps before
8.0 makes strict mode the default.

Additive across the board. Zero behavior change for consumers not using strict
mode. Every change is JS-side — Cortex needs no work for 7.30.1.

NEW — brain.audit() diagnostic
- Read-only method walking storage.getNouns() / getVerbs() pagination
- Returns { entitiesWithoutSubtype: { type: count }, relationshipsWithoutSubtype,
  total, scanned, recommendation }
- VFS infrastructure entities excluded by default (they bypass enforcement via
  isVFSEntity marker); pass { includeVFS: true } to surface them
- The companion to migrateField (7.x) and fillSubtypes (8.0): tells consumers
  exactly what would break under strict enforcement, deterministically

NEW — Improved enforcement error messages
- Caller's source location extracted from Error().stack so users see their own
  call site, not a Brainy internal frame
- Specific guidance branches: registered vocabulary → "Pass one of: a, b, c";
  brain-wide strict mode → mentions the except clause; otherwise → registration
  recipe via brain.requireSubtype()
- Documentation link to the canonical migration recipe
- Same shape for noun and verb enforcement

NEW — CLI --subtype flag
- brainy add and brainy relate gain -s/--subtype <value>
- Defaults to 'cli-add' / 'cli-relate' so the CLI works against strict-mode
  brains without the user needing to know the vocabulary in advance

INTERNAL — every Brainy write path now sets subtype
- VFS Contains edges (5 sites at lines 503/905/1694/1772/1886) → 'vfs-contains'
- VFS symlink entity → 'vfs-symlink' (NEW — distinct from 'vfs-file')
- VFS copy-file → preserves source subtype, falls back to 'vfs-file'
- VFS symlink also adopts the isVFSEntity infrastructure marker so it bypasses
  enforcement in strict mode
- Aggregation materializer (Measurement entities) → 'materialized-aggregate'
- ImportCoordinator (3 sites): document → 'import-source'; entities →
  options.defaultSubtype ?? 'imported'; placeholder → 'import-placeholder'
- SmartImportOrchestrator (4 entity sites + 2 batch relate sites): same
  precedence (extractor → options.defaultSubtype → 'imported')
- EntityDeduplicator → candidate.subtype ?? 'imported'
- UniversalImportAPI → extractor → 'extracted' for both entities and relations
- NeuralImport → adds defaultSubtype to NeuralImportOptions; precedence same
- GoogleSheetsIntegration → request body 'subtype' ?? 'imported-from-sheets'
- ODataIntegration → request body 'Subtype' ?? 'imported-from-odata'
- MCP client message storage → 'mcp-message' (also fixes pre-existing missing
  data field and missing type by aliasing from the prior text field)

Side-effect fix: storage.getNouns() paginated now surfaces subtype to top-level
- Single-noun getNoun() already did this in 7.30; the paginated path was missed
- Without this fix brain.audit() saw missing subtype on entities that actually
  had one (caught by the strict-mode self-test before release)

NEW — tests/integration/strict-mode-self-test.test.ts (13 tests)
- Creates a brain under the exact SDK_CORE_VOCABULARY shape Venue hit + brain-
  wide strict mode
- Exercises every internal Brainy path: VFS root + mkdir + writeFile + cp + mv
  + ln + symlink; aggregation engine; audit diagnostic with includeVFS toggle
- Validates error message UX: caller location, vocabulary guidance, brain-wide
  strict mode guidance, off-vocabulary value reporting

Docs
- New "Strict mode in practice" section in docs/guides/subtypes-and-facets.md
  covering the SDK_CORE_VOCABULARY pattern, 4-step migration recipe
  (audit → migrateField → hand-fix → re-audit), the Brainy-internal label
  reference table, and an 8.0 forward-look on fillSubtypes()
- docs/api/README.md: new audit() entry, strict-mode tips on add() and relate()
- RELEASES.md: full 7.30.1 entry

Cortex parity (forward-looking, not blocking 7.30.1)
- 6th open question added to .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md: native
  fast path for audit() and fillSubtypes() via column-store null-subtype
  bitmap for billion-scale brains
- Cortex should add a parity test mirroring strict-mode-self-test.test.ts
  against their native paths to catch any latent bug where native writes
  bypass JS validation
- Brainy-internal subtype labels become a documented part of the 8.0 contract
  (useful for Cortex telemetry surfacing Brainy-managed infrastructure %)

Verification
- npx tsc --noEmit: clean
- npm test: 1468/1468 unit
- 7.29 noun integration suite: 26/26 (no regression)
- 7.30 verb subtype + enforcement integration suite: 30/30 (no regression)
- New strict-mode-self-test integration suite: 13/13
- npm run build: clean
- Closed-source product reference audit: clean

Addresses VE-SUBTYPE-MIGRATION (Venue's reported request) and ships internal
labels Venue did NOT ask for but that would have broken them next under their
own vocabulary registration.
2026-06-08 11:31:47 -07:00
..
api fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors 2026-06-08 11:31:47 -07:00
architecture feat: verb subtype + updateRelation + requireSubtype enforcement 2026-06-05 11:15:52 -07:00
augmentations fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
concepts docs: storage-adapter inheritance contract + correct the hasStorageMethod story 2026-05-15 13:20:18 -07:00
deployment fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
features fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
guides fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors 2026-06-08 11:31:47 -07:00
operations fix: eliminate cloud storage write amplification and rate limiting 2026-01-31 09:09:36 -08:00
vfs fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
BATCHING.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00
CREATING-AUGMENTATIONS.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
DATA_MODEL.md feat: verb subtype + updateRelation + requireSubtype enforcement 2026-06-05 11:15:52 -07:00
DEVELOPER_LEARNING_PATH.md feat: enforce data/metadata separation, numeric range queries, improved docs 2026-02-09 12:07:54 -08:00
eli5.md docs: replace ASCII box art with prose in Before/After section 2026-02-23 15:59:59 -08:00
EXTENDING_STORAGE.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00
FIND_SYSTEM.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00
MIGRATION-V3-TO-V4.md chore(release): 4.0.0 2025-10-17 14:48:34 -07:00
neural-extraction.md feat: queryAggregate() + HAVING, plus aggregate backfill, traversal depth/via, extraction typing (BR-ADV-FEATURES-BUN) 2026-05-26 13:55:43 -07:00
PERFORMANCE.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
PLUGINS.md docs: add public frontmatter to docs for soulcraft.com/docs pipeline 2026-02-19 17:04:05 -08:00
PRODUCTION_SERVICE_ARCHITECTURE.md feat: migrate embeddings to Candle WASM + remove semantic type inference 2026-01-06 12:52:34 -08:00
QUERY_OPERATORS.md feat: verb subtype + updateRelation + requireSubtype enforcement 2026-06-05 11:15:52 -07:00
README.md feat: enforce data/metadata separation, numeric range queries, improved docs 2026-02-09 12:07:54 -08:00
RELEASE-GUIDE.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
SCALING.md docs: add comprehensive scaling and storage architecture documentation 2025-09-08 14:49:25 -07:00
STAGE3-CANONICAL-TAXONOMY.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
transactions.md fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
troubleshooting.md feat: migrate embeddings to Candle WASM + remove semantic type inference 2026-01-06 12:52:34 -08:00
universal-display-augmentation.md fix: update all imports and references from BrainyData to Brainy 2025-09-30 17:09:15 -07:00

Brainy Documentation

The multi-dimensional AI database with Triple Intelligence — vector search, graph traversal, and metadata filtering in one unified API.

Quick Start

import { Brainy, NounType, VerbType } from '@soulcraft/brainy'

const brain = new Brainy()
await brain.init()

// Add entities — data is embedded for semantic search, metadata is indexed for filtering
const id = await brain.add({
  data: 'Revolutionary AI Breakthrough',
  type: NounType.Document,
  metadata: { category: 'technology', rating: 4.8 }
})

// Search with Triple Intelligence
const results = await brain.find({
  query: 'artificial intelligence',              // Semantic search (on data)
  where: { rating: { greaterThan: 4.0 } },       // Metadata filter
  connected: { from: authorId, depth: 2 }         // Graph traversal
})

Core Documentation

Document Description
API Reference Complete API documentation — start here
Data Model Entity structure, data vs metadata, storage fields
Query Operators All BFO operators with examples and indexed/in-memory matrix
Find System Natural language find() and hybrid search details

Architecture

Document Description
Architecture Overview High-level system design
Triple Intelligence Vector + Graph + Metadata unified query
Noun-Verb Taxonomy 42 nouns + 127 verbs type system
Stage 3 Canonical Taxonomy Complete type reference
Storage Architecture Storage adapters and optimization
Index Architecture HNSW, Graph, and Metadata indexing
Zero Configuration Auto-adapts to any environment

Virtual Filesystem (VFS)

Document Description
VFS Quick Start Get started in 30 seconds
VFS Core Core concepts and architecture
VFS API Guide Complete VFS API reference
Common Patterns VFS usage patterns

See vfs/ for the complete VFS documentation set.


Guides

Document Description
Import Anything CSV, Excel, PDF, URL imports
Natural Language Query in plain English
Neural API AI-powered features
Enterprise for Everyone No limits, no tiers
Framework Integration React, Vue, Angular, Svelte

Storage & Deployment

Document Description
Cloud Deployment Deploy on AWS, GCP, Azure, Cloudflare
Extending Storage Create custom storage adapters
AWS S3 Cost Optimization 96% cost savings
GCS Cost Optimization 94% savings with Autoclass
Azure Cost Optimization 95% savings
R2 Cost Optimization Zero egress fees
Capacity Planning Scale to millions of entities

Plugins & Augmentations

Document Description
Plugins Plugin system overview
Creating Augmentations Build custom plugins
Augmentations Reference Full augmentation API
Augmentations Developer Guide Plugin development guide

Performance & Scaling

Document Description
Performance Optimization techniques
Scaling Scale to billions of entities
Batching Batch operations guide

Migration & Reference

Document Description
v3 to v4 Migration Upgrade guide
Release Guide How to release new versions
Production Architecture Ops reference

Internal

Document Description
Audit Report Feature audit
Honest Status Actual implementation status

License

Brainy is MIT licensed. See LICENSE for details.