David Snelling 1f7e365a4e chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase

2026-06-11 14:51:00 -07:00

10 KiB

Raw Blame History

Data Model

How Brainy stores entities and relationships, and the critical distinction between data and metadata.

Entity (Noun)

An entity is the fundamental data unit in Brainy. Every entity has:

Field	Type	Indexed	Description
`id`	`string`	Primary key	UUID v4 (auto-generated or custom)
`data`	`any`	HNSW vector index	Content used for semantic/hybrid search. Strings auto-embed.
`metadata`	`object`	MetadataIndex	Structured queryable fields (tags, dates, flags, etc.)
`type`	`NounType`	MetadataIndex (as `noun`)	Entity type classification
`vector`	`number[]`	HNSW	384-dim embedding (auto-computed from `data` or user-provided)
`confidence`	`number`	MetadataIndex	Type classification confidence (0-1)
`weight`	`number`	MetadataIndex	Entity importance/salience (0-1)
`service`	`string`	MetadataIndex	Multi-tenancy identifier
`createdAt`	`number`	MetadataIndex	Creation timestamp (ms since epoch)
`updatedAt`	`number`	MetadataIndex	Last update timestamp (ms since epoch)
`createdBy`	`object`	MetadataIndex	Source augmentation info

Example

const id = await brain.add({
  data: 'John Smith is a software engineer at Acme Corp',  // → embedded into vector
  type: NounType.Person,
  metadata: {                   // → indexed, queryable via where filters
    role: 'engineer',
    department: 'backend',
    yearsExperience: 8
  },
  confidence: 0.95,
  weight: 0.7
})

Relationship (Verb)

A relationship is a typed, directed edge connecting two entities.

Field	Type	Indexed	Description
`id`	`string`	Primary key	UUID v4 (auto-generated)
`from`	`string`	GraphAdjacencyIndex	Source entity ID
`to`	`string`	GraphAdjacencyIndex	Target entity ID
`type`	`VerbType`	GraphAdjacencyIndex (as `verb`)	Relationship type classification
`data`	`any`	—	Opaque content (overrides auto-computed vector if provided)
`metadata`	`object`	—	Structured fields on the edge
`weight`	`number`	—	Connection strength (0-1, default: 1.0)
`confidence`	`number`	—	Relationship certainty (0-1)
`evidence`	`RelationEvidence`	—	Why this relationship was detected
`createdAt`	`number`	—	Creation timestamp (ms since epoch)
`updatedAt`	`number`	—	Last update timestamp (ms since epoch)
`service`	`string`	—	Multi-tenancy identifier

Example

const relId = await brain.relate({
  from: personId,
  to: projectId,
  type: VerbType.WorksOn,
  data: 'Lead engineer on the AI module',   // Optional: content for this edge
  metadata: {                                // Optional: queryable edge fields
    role: 'lead',
    startDate: '2024-01-15'
  },
  weight: 0.9
})

Data vs Metadata

This is the most important concept in Brainy's storage model:

`data` — Content for Semantic Search

Embedded into a 384-dimensional vector via the WASM embedding engine
Searchable via semantic similarity (HNSW vector index) and hybrid text+semantic search

Queried by passing query to find():

brain.find({ query: 'machine learning algorithms' })

NOT indexed by MetadataIndex — you cannot use where filters on data
Stored opaquely: strings, objects, numbers — anything goes

`metadata` — Structured Queryable Fields

Indexed by MetadataIndex with O(1) lookups per field

Queryable via where filters using BFO operators:

brain.find({
  where: {
    department: 'engineering',
    yearsExperience: { greaterThan: 5 },
    tags: { contains: 'senior' }
  }
})

NOT used for vector/semantic search
Must be a flat or lightly nested object

Quick Reference

	`data`	`metadata`
Purpose	Content for embedding / semantic search	Structured fields for filtering
Searched by	`find({ query })` — vector similarity, hybrid text+semantic	`find({ where })` — exact, range, set operators
Indexed by	HNSW vector index	MetadataIndex
Queryable with operators?	No	Yes (`equals`, `greaterThan`, `oneOf`, etc.)
Auto-embedded?	Yes (strings → 384-dim vectors)	No
Typical content	Text descriptions, document content	Tags, dates, status flags, categories, numeric fields

Common Pattern

// Add an article
await brain.add({
  data: 'A deep dive into transformer architectures and attention mechanisms',
  type: NounType.Document,
  metadata: {
    title: 'Transformer Deep Dive',
    author: 'Dr. Chen',
    publishedYear: 2024,
    tags: ['AI', 'transformers', 'NLP'],
    status: 'published'
  }
})

// Search by content (semantic — searches data)
const results = await brain.find({ query: 'neural network attention' })

// Filter by fields (exact — queries metadata)
const recent = await brain.find({
  where: {
    publishedYear: { greaterThan: 2023 },
    status: 'published'
  }
})

// Combine both (Triple Intelligence)
const precise = await brain.find({
  query: 'attention mechanisms',             // Semantic search on data
  where: { author: 'Dr. Chen' },            // Metadata filter
  connected: { from: authorId, depth: 1 }   // Graph traversal
})

Storage Field Naming

Internally, Brainy uses different field names in storage vs the public API:

Public API (Entity/Relation)	Storage (metadata object)	Notes
`type`	`noun`	Entity type stored as `noun`
`from`	`sourceId`	Relationship source
`to`	`targetId`	Relationship target
`type` (on Relation)	`verb`	Relationship type stored as `verb`

When querying with find(), you can use:

type parameter (convenience alias, equivalent to where.noun)
where.noun directly

// These are equivalent:
brain.find({ type: NounType.Person })
brain.find({ where: { noun: NounType.Person } })

Standard Metadata Fields

When you add an entity, Brainy stores these standard fields in the metadata object alongside your custom fields:

Field	Set By	Description
`noun`	System	Entity type (NounType enum value)
`subtype`	User	Per-NounType sub-classification (e.g. `'employee'`, `'invoice'`, `'milestone'`). Flat string, no hierarchy. Indexed on the fast path and rolled into per-NounType statistics.
`data`	System	The raw `data` value (stored opaquely)
`createdAt`	System	Creation timestamp
`updatedAt`	System	Last update timestamp
`confidence`	User	Type classification confidence
`weight`	User	Entity importance
`service`	User	Multi-tenancy identifier
`createdBy`	User/System	Source augmentation

On read, these standard fields are extracted to top-level Entity properties. The metadata field on the returned Entity contains only your custom fields.

Subtype — sub-classification within a NounType

type (NounType) is a stable 42-value enum. subtype is the consumer-chosen string vocabulary within a type:

// A Person who is an employee:
await brain.add({
  data: 'Avery Brooks — runs the AI lab',
  type: NounType.Person,
  subtype: 'employee',
  metadata: { department: 'ai-lab' }
})

// A Document that is an invoice:
await brain.add({
  data: 'INV-2026-001',
  type: NounType.Document,
  subtype: 'invoice',
  metadata: { amount: 1500 }
})

subtype lives at the top level — NOT inside metadata, NOT inside data. That's how find({ type, subtype }) routes through the standard-field fast path (column-store hit) instead of the metadata fallback. See Subtypes & Facets for the full guide including trackField() and migrateField().

Subtype — sub-classification within a VerbType (7.30+)

Relationships are first-class citizens too. Every verb (VerbType) gets the same subtype primitive — a ReportsTo relationship might carry subtype: 'direct' vs 'dotted-line'; a RelatedTo edge might carry 'spouse' / 'sibling' / 'colleague'. Same shape as the noun side: flat string, no hierarchy, top-level standard field on HNSWVerbWithMetadata and on the public Relation<T>:

await brain.relate({
  from: ceoId,
  to: vpId,
  type: VerbType.ReportsTo,
  subtype: 'direct',                  // top-level standard field
  metadata: { since: '2025-Q1' }      // user-custom fields stay in metadata
})

Fast-path filter on the verb side:

const direct = await brain.related({
  from: ceoId,
  type: VerbType.ReportsTo,
  subtype: 'direct'
})

The verb-side rollup at _system/verb-subtype-statistics.json mirrors the noun-side _system/subtype-statistics.json — same shape, same self-heal machinery. Per-VerbType-per-subtype counts are O(1) via brain.counts.byRelationshipSubtype().

Verbs and nouns now have full capability parity — every API on the noun side has a verb-side mirror, including the new brain.updateRelation() (which closed a pre-7.30 gap where relationships had no update path).

Standard verb fields

The verb-side equivalent of STANDARD_ENTITY_FIELDS is STANDARD_VERB_FIELDS, exported from src/coreTypes.ts. Verb-specific standard fields:

Field	Description
`verb`	The VerbType enum value
`sourceId` / `targetId`	The two endpoints of the relationship
`subtype`	Sub-classification within the VerbType (7.30+)
`confidence`, `weight`, `createdAt`, `updatedAt`, `service`, `createdBy`, `data`	Same semantics as the noun-side standard fields

The companion resolveVerbField(verb, field) helper resolves field paths the same way resolveEntityField does for nouns: standard fields first, metadata fallback for everything else.

10 KiB Raw Blame History

Data Model

Entity (Noun)

Example

Relationship (Verb)

Example

Data vs Metadata

data — Content for Semantic Search

metadata — Structured Queryable Fields

Quick Reference

Common Pattern

Storage Field Naming

Standard Metadata Fields

Subtype — sub-classification within a NounType

Subtype — sub-classification within a VerbType (7.30+)

Standard verb fields

See Also

10 KiB

Raw Blame History

`data` — Content for Semantic Search

`metadata` — Structured Queryable Fields