chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase

This commit is contained in:
David Snelling 2026-06-11 14:51:00 -07:00
parent 970e08c466
commit 1f7e365a4e
237 changed files with 1951 additions and 49413 deletions

View file

@ -185,12 +185,12 @@ await brain.update({
---
### `delete(id)` → `Promise<void>`
### `remove(id)` → `Promise<void>`
Delete a single entity.
Remove a single entity (and every relationship where it is source or target).
```typescript
await brain.delete(id)
await brain.remove(id)
```
**Parameters:**
@ -444,7 +444,7 @@ brain.defineAggregate({
name: 'monthly_spending',
source: {
type: NounType.Event,
where: { domain: 'financial', subtype: 'transaction' }
where: { domain: 'financial' } // matches custom metadata fields
},
groupBy: [
'category',
@ -468,10 +468,10 @@ brain.defineAggregate({
|-------|------|-------------|
| `name` | `string` | Unique identifier for this aggregate |
| `source.type` | `NounType \| NounType[]` | Entity types that feed into this aggregate |
| `source.where` | `Record<string, unknown>` | Metadata filter (same syntax as `find({ where })`) |
| `source.where` | `Record<string, unknown>` | Filter on custom **metadata** fields (matched against the entity's `metadata` bag) |
| `source.service` | `string` | Multi-tenancy filter |
| `groupBy` | `GroupByDimension[]` | Dimensions to group by — plain field names or `{ field, window }` for time bucketing |
| `metrics` | `Record<string, AggregateMetricDef>` | Named metrics with `op` (`sum`, `count`, `avg`, `min`, `max`, `stddev`, `variance`) and optional `field` |
| `groupBy` | `GroupByDimension[]` | Dimensions to group by — plain field names, `{ field, window }` for time bucketing, or `{ field, unnest: true }` for array fields (one contribution per element) |
| `metrics` | `Record<string, AggregateMetricDef>` | Named metrics with `op` (`sum`, `count`, `avg`, `min`, `max`, `stddev`, `variance`, `percentile`, `distinctCount`) and optional `field`. `percentile` additionally requires `p` in `[0, 1]`. |
| `materialize` | `boolean \| object` | Write results as `NounType.Measurement` entities (auto-visible in OData/Sheets/SSE) |
**Time window granularities:** `'hour'`, `'day'`, `'week'`, `'month'`, `'quarter'`, `'year'`, or `{ seconds: number }` for custom intervals.
@ -539,6 +539,20 @@ const recentFood = await brain.find({
}
```
### `queryAggregate(name, params?)``Promise<AggregateResult[]>`
The first-class analytics path — returns plain group rows (`{ groupKey, metrics, count }`) instead of `find()`-style `Result` wrappers. Supports `where` (group-key filter), `having` (SQL-HAVING metric filter), `orderBy`, `order`, `limit`, `offset`:
```typescript
const rows = await brain.queryAggregate('monthly_spending', {
having: { total: { greaterThan: 100 } }, // filter by computed metrics
orderBy: 'total',
order: 'desc',
limit: 10
})
// [{ groupKey: { category: 'food', date: '2024-01' }, metrics: { total: 342.5, count: 28 }, count: 28 }, ...]
```
### How It Works
Aggregation hooks run **outside transactions** on every write operation:
@ -557,16 +571,16 @@ Aggregation hooks run **outside transactions** on every write operation:
### Financial Data Modeling
Brainy supports financial analytics through **metadata conventions** on existing NounTypes — no custom types needed:
Brainy supports financial analytics through **subtypes and metadata conventions** on existing NounTypes — no custom types needed:
```typescript
// Transaction = NounType.Event + financial metadata
// Transaction = NounType.Event + 'transaction' subtype + financial metadata
await brain.add({
data: 'Coffee at Blue Bottle',
type: NounType.Event,
subtype: 'transaction', // top-level standard field (reserved — never in metadata)
metadata: {
domain: 'financial',
subtype: 'transaction',
amount: 5.50,
currency: 'USD',
category: 'food',
@ -575,26 +589,26 @@ await brain.add({
}
})
// Account = NounType.Collection + financial metadata
// Account = NounType.Collection + 'account' subtype + financial metadata
await brain.add({
data: 'Checking Account',
type: NounType.Collection,
subtype: 'account',
metadata: {
domain: 'financial',
subtype: 'account',
accountType: 'checking',
currency: 'USD',
institution: 'Chase'
}
})
// Invoice = NounType.Document + financial metadata
// Invoice = NounType.Document + 'invoice' subtype + financial metadata
await brain.add({
data: 'Invoice #1234 from Acme Corp',
type: NounType.Document,
subtype: 'invoice',
metadata: {
domain: 'financial',
subtype: 'invoice',
amount: 15000,
currency: 'USD',
status: 'pending',
@ -672,32 +686,33 @@ await brain.updateRelation({ id: relId, type: VerbType.WorksWith })
---
### `getRelations(params)` → `Promise<Relation[]>`
### `related(params)` → `Promise<Relation[]>`
Get relationships for an entity.
Get relationships for an entity. Same name and surface as `db.related()` on a
pinned `Db` view.
```typescript
// Get all relationships FROM an entity
const outgoing = await brain.getRelations({ from: entityId })
const outgoing = await brain.related({ from: entityId })
// Get all relationships TO an entity
const incoming = await brain.getRelations({ to: entityId })
const incoming = await brain.related({ to: entityId })
// Filter by type
const related = await brain.getRelations({
const contains = await brain.related({
from: entityId,
type: VerbType.Contains
})
// Filter by subtype (fast path, column-store hit)
const direct = await brain.getRelations({
const direct = await brain.related({
from: entityId,
type: VerbType.ReportsTo,
subtype: 'direct'
})
// Set membership on subtype
const all = await brain.getRelations({
const all = await brain.related({
from: entityId,
type: VerbType.ReportsTo,
subtype: ['direct', 'dotted-line']
@ -739,12 +754,12 @@ console.log(result.failed) // Array of errors
---
### `deleteMany(params)` → `Promise<BatchResult<string>>`
### `removeMany(params)` → `Promise<BatchResult<string>>`
Delete multiple entities.
Remove multiple entities.
```typescript
const result = await brain.deleteMany({
const result = await brain.removeMany({
ids: [id1, id2, id3]
})
```
@ -757,7 +772,7 @@ Update multiple entities.
```typescript
const result = await brain.updateMany({
updates: [
items: [
{ id: id1, metadata: { updated: true } },
{ id: id2, data: 'New content' }
]
@ -772,7 +787,7 @@ Create multiple relationships.
```typescript
const ids = await brain.relateMany({
relations: [
items: [
{ from: id1, to: id2, type: VerbType.RelatedTo },
{ from: id1, to: id3, type: VerbType.Contains }
]
@ -1213,39 +1228,16 @@ const todos = await brain.vfs.getTodos('/src/App.tsx')
---
#### `vfs.getAllTodos(path?)` → `Promise<Todo[]>`
#### `vfs.searchEntities(query)` → `Promise<Array<{ id, path, type, metadata }>>`
Get all TODOs from directory tree.
Search for semantic entities tracked by the VFS, filtered by type, name, or metadata.
```typescript
const allTodos = await brain.vfs.getAllTodos('/src')
```
---
### Project Analysis
#### `vfs.getProjectStats(path?)``Promise<Stats>`
Get project statistics.
```typescript
const stats = await brain.vfs.getProjectStats('/projects/my-app')
console.log(stats.fileCount)
console.log(stats.totalSize)
console.log(stats.fileTypes) // Breakdown by extension
```
---
#### `vfs.searchEntities(query)``Promise<VFSEntity[]>`
Search for VFS entities by metadata.
```typescript
const tsxFiles = await brain.vfs.searchEntities({
type: 'file',
extension: '.tsx'
const people = await brain.vfs.searchEntities({
type: 'person', // entity type filter
name: 'Ada', // semantic name search
where: { role: 'author' }, // metadata filters
limit: 50
})
```
@ -1280,48 +1272,52 @@ console.log(result.explanation)
---
### `neural().clusters(input?, options?)``Promise<Cluster[]>`
### `neural().clusters(input?, options?)``Promise<SemanticCluster[]>`
Automatic clustering.
Automatic clustering. Accepts a text query, an array of texts, or a `ClusteringOptions` object.
```typescript
const clusters = await brain.neural().clusters({
algorithm: 'kmeans',
k: 5,
minSize: 3
algorithm: 'kmeans', // 'auto' | 'hierarchical' | 'kmeans' | 'dbscan' | ...
maxClusters: 5,
minClusterSize: 3
})
clusters.forEach(cluster => {
console.log(cluster.label)
console.log(cluster.items)
console.log(cluster.centroid)
console.log(cluster.label) // auto-generated label (when available)
console.log(cluster.members) // entity ids in the cluster
console.log(cluster.centroid) // centroid vector
})
```
---
### `neural().neighbors(id, options?)``Promise<Neighbor[]>`
### `neural().neighbors(id, options?)``Promise<NeighborsResult>`
Find k-nearest neighbors.
Find nearest neighbors.
```typescript
const neighbors = await brain.neural().neighbors(entityId, {
k: 10,
threshold: 0.7
const result = await brain.neural().neighbors(entityId, {
limit: 10,
minSimilarity: 0.7
})
```
**Options:** `limit?`, `radius?`, `minSimilarity?`, `includeMetadata?`, `sortBy?: 'similarity' | 'importance' | 'recency'`
---
### `neural().outliers(threshold?)` → `Promise<string[]>`
### `neural().outliers(options?)` → `Promise<Outlier[]>`
Detect outlier entities.
```typescript
const outliers = await brain.neural().outliers(0.3)
// Returns entity IDs that are outliers
const outliers = await brain.neural().outliers({ threshold: 0.3 })
// Each outlier carries the entity id plus outlier scoring details
```
**Options:** `threshold?`, `method?: 'isolation' | 'statistical' | 'cluster-based'`, `minNeighbors?`, `includeReasons?`
---
### `neural().visualize(options?)``Promise<VizData>`
@ -1342,27 +1338,28 @@ const vizData = await brain.neural().visualize({
### Performance Methods
#### `neural().clusterFast(options)` → `Promise<Cluster[]>`
#### `neural().clusterFast(options?)` → `Promise<SemanticCluster[]>`
Fast clustering for large datasets.
```typescript
const clusters = await brain.neural().clusterFast({
k: 10,
maxIterations: 50
maxClusters: 10
})
```
**Options:** `level?` (hierarchy level), `maxClusters?`
---
#### `neural().clusterLarge(options)` → `Promise<Cluster[]>`
#### `neural().clusterLarge(options?)` → `Promise<SemanticCluster[]>`
Streaming clustering for very large datasets.
Sampling-based clustering for very large datasets.
```typescript
const clusters = await brain.neural().clusterLarge({
k: 20,
batchSize: 1000
sampleSize: 1000,
strategy: 'diverse' // 'random' | 'diverse' | 'recent'
})
```
@ -1381,17 +1378,15 @@ await brain.import('data.csv', {
createEntities: true
})
// Excel import
// Excel import (all sheets processed automatically)
await brain.import('sales.xlsx', {
format: 'excel',
sheets: ['Q1', 'Q2']
vfsPath: '/imports/sales', // optional: mirror into the VFS
groupBy: 'sheet'
})
// PDF import
await brain.import('research.pdf', {
format: 'pdf',
extractTables: true
})
// PDF import (tables extracted automatically)
await brain.import('research.pdf', { format: 'pdf' })
// URL import
await brain.import('https://api.example.com/data.json')
@ -1400,10 +1395,17 @@ await brain.import('https://api.example.com/data.json')
**Parameters:**
- `source`: `string | Buffer | object` - File path, URL, buffer, or object
- `options?`: Import configuration
- `format?`: `'csv' | 'excel' | 'pdf' | 'json'` - Auto-detected if omitted
- `format?`: `'excel' | 'pdf' | 'csv' | 'json' | 'markdown' | 'yaml' | 'docx' | 'image'` - Auto-detected if omitted
- `vfsPath?`: `string` - Mirror imported content into the VFS at this path
- `groupBy?`: `'type' | 'sheet' | 'flat' | 'custom'` - VFS grouping strategy
- `createEntities?`: `boolean` - Create entities from rows
- `sheets?`: `string[]` - Excel sheets to import
- `extractTables?`: `boolean` - Extract tables from PDF
- `createRelationships?`: `boolean` - Create relationships between extracted entities
- `preserveSource?`: `boolean` - Save the original file in the VFS
- `enableNeuralExtraction?`: `boolean` - Extract entity names via AI
- `enableRelationshipInference?`: `boolean` - Infer relationships via AI
- `enableConceptExtraction?`: `boolean` - Extract entity types via AI
- `confidenceThreshold?`: `number` - Minimum confidence for extracted entities
- `onProgress?`: `(progress) => void` - Progress callback (stage, counts, throughput, ETA)
**Returns:** `Promise<ImportResult>` - Import statistics
@ -1414,9 +1416,6 @@ await brain.import('https://api.example.com/data.json')
### Export & Snapshots
```typescript
// Export to file
await brain.export('/path/to/backup.brainy')
// Instant hard-link snapshot via the Db API
const pin = brain.now()
await pin.persist('/backups/2026-06-11')
@ -1453,12 +1452,11 @@ const brain = new Brainy({
// Model: all-MiniLM-L6-v2 (384 dimensions)
// Device: CPU via WASM (works everywhere)
// Cache configuration
cache: {
enabled: true,
maxSize: 10000,
ttl: 3600000 // 1 hour in ms
}
// Cache configuration — `true`/`false`, or an options object
cache: {
maxSize: 10000,
ttl: 3600000 // 1 hour in ms
}
})
await brain.init() // Required! VFS auto-initialized
@ -1672,7 +1670,7 @@ When strict mode is on:
- Brainy's own VFS infrastructure writes bypass via the `metadata.isVFSEntity: true` marker.
- Per-type registrations always apply regardless of the brain-wide flag.
Becomes the default in 8.0.0.
The default since 8.0.0 — pass `requireSubtype: false` to opt out while migrating pre-8.0 data.
#### `trackField(name, options?)``void`
@ -1718,6 +1716,27 @@ await brain.migrateField({
Returns `{ scanned: number, migrated: number, skipped: number, errors: Array<{id, error}> }`.
#### `fillSubtypes(rules, options?)``Promise<FillSubtypesResult>` (8.0+)
Back-fill missing `subtype` values across entities AND relationships in one streaming pass — the migration companion to `audit()`. Keys are `NounType`/`VerbType` values; each rule is a literal subtype string or a function deriving one from the entry (return `undefined` to decline). Idempotent: entries that already carry a subtype are never touched, so a crashed run is resumed safely by re-running.
```typescript
const report = await brain.fillSubtypes({
[NounType.Person]: (e) => e.metadata?.kind ?? 'unspecified', // derived
[NounType.Document]: 'general', // literal default
[VerbType.RelatedTo]: 'unspecified' // relationship rule
})
// → { scanned, filled, skipped, errors, byType }
```
**Parameters:**
- `rules`: `FillSubtypeRules` - Map of NounType/VerbType → literal subtype or `(entry) => string | undefined`
- `options.includeVFS?`: `boolean` - Also fill VFS infrastructure entries (default `false`)
- `options.batchSize?`: `number` - Pagination batch size (default `200`)
- `options.onProgress?`: `(progress: { scanned, filled, skipped }) => void` - Per-batch callback
Returns `{ scanned, filled, skipped, errors, byType }`. After a clean run, `skipped` equals the remaining `audit().total`. See the [migration recipe](../guides/subtypes-and-facets.md).
---
### `embed(data)``Promise<number[]>`
@ -1812,7 +1831,7 @@ for (const group of duplicates) {
// Find person duplicates with higher threshold
const personDupes = await brain.findDuplicates({
type: NounType.PERSON,
type: NounType.Person,
threshold: 0.9,
limit: 50
})
@ -1885,15 +1904,19 @@ const docClusters = await brain.cluster({
---
### `getStats()` → `Statistics`
### `getStats(options?)` → `Promise<Statistics>`
Get comprehensive statistics.
Get complete entity/relationship statistics (convenience wrapper over `brain.counts`).
```typescript
const stats = brain.getStats()
console.log(stats.entityCount)
console.log(stats.relationshipCount)
console.log(stats.cacheHitRate)
const stats = await brain.getStats()
console.log(stats.entities.total) // total entity count
console.log(stats.entities.byType) // counts per NounType
console.log(stats.relationships) // relationship stats
console.log(stats.density) // relationships per entity
// Exclude VFS infrastructure entities from the counts
const semanticOnly = await brain.getStats({ excludeVFS: true })
```
---
@ -1914,7 +1937,7 @@ VFS is auto-initialized during `brain.init()` - no separate `vfs.init()` needed!
### Shutdown
```typescript
await brain.shutdown() // Graceful shutdown, flush caches
await brain.close() // Graceful shutdown — flushes pending writes and releases the writer lock
```
---
@ -1940,8 +1963,8 @@ await brain.update({
metadata: { updated: true }
})
// Delete
await brain.delete(id)
// Remove
await brain.remove(id)
```
---
@ -2001,7 +2024,7 @@ const results = await brain.find({
// Speculate: what would this change look like? (nothing touches disk)
const base = brain.now()
const whatIf = await base.with([
{ op: 'add', type: NounType.Document, subtype: 'note', data: 'New feature' }
{ op: 'add', type: NounType.Document, subtype: 'note', data: 'New feature', metadata: { draft: true } }
])
await whatIf.find({ where: { draft: true } })
await whatIf.release()
@ -2009,7 +2032,7 @@ await base.release()
// Commit it for real — one atomic generation, with audit metadata
await brain.transact(
[{ op: 'add', type: NounType.Document, subtype: 'note', data: 'New feature' }],
[{ op: 'add', type: NounType.Document, subtype: 'note', data: 'New feature', metadata: { draft: true } }],
{ meta: { author: 'dev@example.com', message: 'Add new feature' } }
)
```
@ -2115,7 +2138,8 @@ For the full taxonomy with all 169 types and their descriptions, see:
- **[Stage 3 CANONICAL Taxonomy](../STAGE3-CANONICAL-TAXONOMY.md)** - Complete list with categories
- **[Noun-Verb Taxonomy Architecture](../architecture/noun-verb-taxonomy.md)** - Design rationale
### Migration from
### Migration from pre-Stage-3 taxonomies
**Breaking Changes:**
- `NounType.Content` removed → Use `Document`, `Message`, or `InformationContent`
- `NounType.User` removed → Use `Person` or `Agent`
@ -2161,7 +2185,6 @@ For the full taxonomy with all 169 types and their descriptions, see:
- **[Find System](../FIND_SYSTEM.md)** - Natural language find() details
- **[VFS Quick Start](../vfs/QUICK_START.md)** - Complete VFS documentation
- **[Import Anything Guide](../guides/import-anything.md)** - CSV, Excel, PDF, URL imports
- **[Cloud Deployment](../deployment/CLOUD_DEPLOYMENT_GUIDE.md)** - Production deployment
- **[Consistency Model](../concepts/consistency-model.md)** - The guarantees behind the Db API
- **[Snapshots & Time Travel](../guides/snapshots-and-time-travel.md)** - Backup, restore, what-if, audit recipes