fix: internal subtype consistency + brain.audit() diagnostic + improved enforcement errors
Brainy 7.30 shipped opt-in subtype enforcement; SDK 3.20.0 then registered
SDK_CORE_VOCABULARY on every consumer's brain (Event, Collection, Message,
Contract, Media, Document NounTypes). On 2026-06-08 Venue's /book flow went 500
because their brain.add({ type: NounType.Event, ... }) call sites lacked
subtype. An audit of Brainy's OWN source revealed 14 HIGH-risk internal write
paths that also omit subtype — any consumer running the same vocabulary would
have hit Brainy's infrastructure paths next. 7.30.1 closes both gaps before
8.0 makes strict mode the default.
Additive across the board. Zero behavior change for consumers not using strict
mode. Every change is JS-side — Cortex needs no work for 7.30.1.
NEW — brain.audit() diagnostic
- Read-only method walking storage.getNouns() / getVerbs() pagination
- Returns { entitiesWithoutSubtype: { type: count }, relationshipsWithoutSubtype,
total, scanned, recommendation }
- VFS infrastructure entities excluded by default (they bypass enforcement via
isVFSEntity marker); pass { includeVFS: true } to surface them
- The companion to migrateField (7.x) and fillSubtypes (8.0): tells consumers
exactly what would break under strict enforcement, deterministically
NEW — Improved enforcement error messages
- Caller's source location extracted from Error().stack so users see their own
call site, not a Brainy internal frame
- Specific guidance branches: registered vocabulary → "Pass one of: a, b, c";
brain-wide strict mode → mentions the except clause; otherwise → registration
recipe via brain.requireSubtype()
- Documentation link to the canonical migration recipe
- Same shape for noun and verb enforcement
NEW — CLI --subtype flag
- brainy add and brainy relate gain -s/--subtype <value>
- Defaults to 'cli-add' / 'cli-relate' so the CLI works against strict-mode
brains without the user needing to know the vocabulary in advance
INTERNAL — every Brainy write path now sets subtype
- VFS Contains edges (5 sites at lines 503/905/1694/1772/1886) → 'vfs-contains'
- VFS symlink entity → 'vfs-symlink' (NEW — distinct from 'vfs-file')
- VFS copy-file → preserves source subtype, falls back to 'vfs-file'
- VFS symlink also adopts the isVFSEntity infrastructure marker so it bypasses
enforcement in strict mode
- Aggregation materializer (Measurement entities) → 'materialized-aggregate'
- ImportCoordinator (3 sites): document → 'import-source'; entities →
options.defaultSubtype ?? 'imported'; placeholder → 'import-placeholder'
- SmartImportOrchestrator (4 entity sites + 2 batch relate sites): same
precedence (extractor → options.defaultSubtype → 'imported')
- EntityDeduplicator → candidate.subtype ?? 'imported'
- UniversalImportAPI → extractor → 'extracted' for both entities and relations
- NeuralImport → adds defaultSubtype to NeuralImportOptions; precedence same
- GoogleSheetsIntegration → request body 'subtype' ?? 'imported-from-sheets'
- ODataIntegration → request body 'Subtype' ?? 'imported-from-odata'
- MCP client message storage → 'mcp-message' (also fixes pre-existing missing
data field and missing type by aliasing from the prior text field)
Side-effect fix: storage.getNouns() paginated now surfaces subtype to top-level
- Single-noun getNoun() already did this in 7.30; the paginated path was missed
- Without this fix brain.audit() saw missing subtype on entities that actually
had one (caught by the strict-mode self-test before release)
NEW — tests/integration/strict-mode-self-test.test.ts (13 tests)
- Creates a brain under the exact SDK_CORE_VOCABULARY shape Venue hit + brain-
wide strict mode
- Exercises every internal Brainy path: VFS root + mkdir + writeFile + cp + mv
+ ln + symlink; aggregation engine; audit diagnostic with includeVFS toggle
- Validates error message UX: caller location, vocabulary guidance, brain-wide
strict mode guidance, off-vocabulary value reporting
Docs
- New "Strict mode in practice" section in docs/guides/subtypes-and-facets.md
covering the SDK_CORE_VOCABULARY pattern, 4-step migration recipe
(audit → migrateField → hand-fix → re-audit), the Brainy-internal label
reference table, and an 8.0 forward-look on fillSubtypes()
- docs/api/README.md: new audit() entry, strict-mode tips on add() and relate()
- RELEASES.md: full 7.30.1 entry
Cortex parity (forward-looking, not blocking 7.30.1)
- 6th open question added to .strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md: native
fast path for audit() and fillSubtypes() via column-store null-subtype
bitmap for billion-scale brains
- Cortex should add a parity test mirroring strict-mode-self-test.test.ts
against their native paths to catch any latent bug where native writes
bypass JS validation
- Brainy-internal subtype labels become a documented part of the 8.0 contract
(useful for Cortex telemetry surfacing Brainy-managed infrastructure %)
Verification
- npx tsc --noEmit: clean
- npm test: 1468/1468 unit
- 7.29 noun integration suite: 26/26 (no regression)
- 7.30 verb subtype + enforcement integration suite: 30/30 (no regression)
- New strict-mode-self-test integration suite: 13/13
- npm run build: clean
- Closed-source product reference audit: clean
Addresses VE-SUBTYPE-MIGRATION (Venue's reported request) and ships internal
labels Venue did NOT ask for but that would have broken them next under their
own vocabulary registration.
This commit is contained in:
parent
a82c3339df
commit
5f3a2ca7d5
18 changed files with 999 additions and 37 deletions
|
|
@ -131,6 +131,8 @@ const id = await brain.add({
|
|||
|
||||
> **`data`** is embedded into vectors for semantic search. **`metadata`** is indexed for `where` filters. See [Data Model](../DATA_MODEL.md).
|
||||
|
||||
> **Strict-mode tip:** if a vocabulary is registered for your `type` (via `brain.requireSubtype()` or by an SDK that wraps Brainy), you must pass a matching `subtype`. Run `await brain.audit()` to inventory pre-existing gaps before enabling strict mode; see the [migration recipe](../guides/subtypes-and-facets.md#strict-mode-in-practice-for-sdk-style-vocabulary-consumers).
|
||||
|
||||
**Returns:** `Promise<string>` - Entity ID
|
||||
|
||||
---
|
||||
|
|
@ -629,6 +631,8 @@ const relId = await brain.relate({
|
|||
- `bidirectional?`: `boolean` - Create reverse edge too (default: false)
|
||||
- `confidence?`: `number` - Relationship certainty (0-1)
|
||||
|
||||
> **Strict-mode tip:** same as `add()` — if a vocabulary is registered for your `type`, pass a matching `subtype`. Run `await brain.audit()` first to surface pre-existing gaps.
|
||||
|
||||
**Returns:** `Promise<string>` - Relationship ID
|
||||
|
||||
---
|
||||
|
|
@ -2049,6 +2053,28 @@ brain.relationshipSubtypesOf(VerbType.ReportsTo)
|
|||
// → ['direct', 'dotted-line']
|
||||
```
|
||||
|
||||
#### `audit(options?)` → `Promise<AuditReport>` (7.30.1+)
|
||||
|
||||
Diagnostic — find entities and relationships missing a `subtype` value, grouped by type. The companion to `migrateField()` / `fillSubtypes()` — answers "what would break if I enabled strict subtype enforcement?".
|
||||
|
||||
```typescript
|
||||
const report = await brain.audit()
|
||||
// {
|
||||
// entitiesWithoutSubtype: { event: 24, document: 3 },
|
||||
// relationshipsWithoutSubtype: { relatedTo: 1402 },
|
||||
// total: 1429,
|
||||
// scanned: 8400,
|
||||
// recommendation: 'Found 1429 entries without subtype. ...'
|
||||
// }
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `options.includeVFS?`: `boolean` — When `false` (default), VFS infrastructure entities (`metadata.isVFSEntity` / `metadata.isVFS`) are excluded. They bypass enforcement anyway, so counting them is noise.
|
||||
- `options.batchSize?`: `number` — Pagination batch size (default 200).
|
||||
- `options.onProgress?`: `(progress: { scanned, missingSubtype }) => void` — Progress callback per batch.
|
||||
|
||||
Run before adopting an SDK that registers `requireSubtype()` rules, or before upgrading to Brainy 8.0 (which makes strict mode the default). See the [Strict mode in practice](../guides/subtypes-and-facets.md#strict-mode-in-practice-for-sdk-style-vocabulary-consumers) guide for the full migration recipe.
|
||||
|
||||
#### `requireSubtype(type, options?)` → `void`
|
||||
|
||||
Register subtype enforcement for a specific `NounType` or `VerbType`. Unified API for nouns and verbs. Composes with the brain-wide `requireSubtype` constructor flag.
|
||||
|
|
|
|||
|
|
@ -460,6 +460,82 @@ brain.requireSubtype(NounType.Person, {
|
|||
await brain.add({ type: NounType.Person, subtype: 'employee', data: '...' })
|
||||
```
|
||||
|
||||
## Strict mode in practice (for SDK-style vocabulary consumers)
|
||||
|
||||
When a platform layer like the Soulcraft SDK registers `requireSubtype()` rules on behalf of every consumer's brain, every downstream product that calls `brain.add()` / `brain.relate()` against those types must pass a matching `subtype`. Skipping the field — or passing one outside the registered vocabulary — throws at the boundary.
|
||||
|
||||
This pattern is powerful but surfaces a class of latent bug: any `brain.add()` call site that was written before strict-mode adoption starts rejecting writes. The Venue team hit this in production on 2026-06-08 when their `/book` flow 500'd on every request because `BookingDraftService.getOrCreateByToken` called `brain.add({ type: NounType.Event, ... })` without subtype.
|
||||
|
||||
The fix is a four-step migration recipe — and Brainy 7.30.1+ ships diagnostic tools to make it deterministic.
|
||||
|
||||
### Migration recipe
|
||||
|
||||
1. **Inventory the gap with `brain.audit()`** — returns the deterministic list of which NounTypes and VerbTypes have entities/relationships missing subtype, grouped by type:
|
||||
|
||||
```typescript
|
||||
const report = await brain.audit()
|
||||
// {
|
||||
// entitiesWithoutSubtype: { event: 24, document: 3, ... },
|
||||
// relationshipsWithoutSubtype: { relatedTo: 1402 },
|
||||
// total: 1429,
|
||||
// scanned: 8400,
|
||||
// recommendation: 'Found 1429 entries without subtype. ...'
|
||||
// }
|
||||
```
|
||||
|
||||
By default, VFS infrastructure entities are excluded (they bypass enforcement anyway via the `metadata.isVFSEntity` marker). Pass `{ includeVFS: true }` to surface them too.
|
||||
|
||||
2. **Bulk-migrate any existing convention** with `brain.migrateField()` if a legacy field can be lifted:
|
||||
|
||||
```typescript
|
||||
// Venue chose subtype = same string as metadata.entityType:
|
||||
await brain.migrateField({
|
||||
from: 'metadata.entityType',
|
||||
to: 'subtype',
|
||||
readBoth: true // safety: keep the source field readable during cutover
|
||||
})
|
||||
```
|
||||
|
||||
3. **Hand-fix the remaining call sites.** The exact list is in `report.entitiesWithoutSubtype`. For each call site, add `subtype: '<value>'` to the `brain.add()` / `brain.relate()` params. Choose a stable convention (Venue chose `subtype = metadata.entityType`; any rule that's deterministic from the data works).
|
||||
|
||||
4. **Verify with `brain.audit()` again.** Re-run; total should be `0`. If you turn on brain-wide strict mode at this point, all future writes are protected.
|
||||
|
||||
### Brainy's own infrastructure subtype labels (reference)
|
||||
|
||||
Brainy's internal write paths set subtype on every entity and edge they create. Consumers don't need to do anything for these — they're documented here so you understand the data shape:
|
||||
|
||||
| Code path | NounType / VerbType | Subtype label |
|
||||
|---|---|---|
|
||||
| VFS root directory `/` | `NounType.Collection` | `'vfs-root'` |
|
||||
| VFS subdirectories | `NounType.Collection` | `'vfs-directory'` |
|
||||
| VFS files | (mime-driven, e.g. `Document`/`Code`/`Image`) | `'vfs-file'` |
|
||||
| VFS symlinks | `NounType.File` | `'vfs-symlink'` |
|
||||
| VFS Contains edges | `VerbType.Contains` | `'vfs-contains'` |
|
||||
| Aggregation materialized output | `NounType.Measurement` | `'materialized-aggregate'` |
|
||||
| Import-document provenance entity | `NounType.Document` | `'import-source'` |
|
||||
| Importer-extracted entities (no caller default) | extractor-driven | `'imported'` |
|
||||
| Importer placeholder targets | `NounType.Thing` | `'import-placeholder'` |
|
||||
| Neural extraction (no caller default) | extractor-driven | `'extracted'` |
|
||||
| GoogleSheets API entity writes | request-driven | `'imported-from-sheets'` |
|
||||
| OData API entity writes | request-driven | `'imported-from-odata'` |
|
||||
| MCP client message storage | `NounType.Message` | `'mcp-message'` |
|
||||
| `brainy add` CLI (no `--subtype` flag) | user-supplied type | `'cli-add'` |
|
||||
| `brainy relate` CLI (no `--subtype` flag) | user-supplied verb | `'cli-relate'` |
|
||||
|
||||
You can query these directly: `await brain.find({ subtype: 'vfs-file' })` returns every VFS-managed file regardless of NounType. `await brain.counts.bySubtype(NounType.Document)` shows you the import-source / imported / extracted / vfs-file breakdown.
|
||||
|
||||
Importer and extraction paths accept a caller-supplied `defaultSubtype` option so you can tag a whole batch with your own provenance label (e.g. `'customer-upload-2026q2'`) instead of the Brainy default `'imported'` / `'extracted'`.
|
||||
|
||||
### Looking ahead — Brainy 8.0
|
||||
|
||||
Brainy 8.0 ships:
|
||||
|
||||
- **`brain.fillSubtypes(rules)`** — the bulk migration helper that pairs with `audit()`. Given caller-supplied rules per NounType / VerbType, it walks the brain and fills in missing subtypes via `update()`. Pre-8.0 brains run this once before upgrading to clear migration debt.
|
||||
- **`subtype: string` (non-optional)** on `AddParams<T>` and `RelateParams<T>`. TypeScript catches missing subtype at compile time, not just runtime.
|
||||
- **`new Brainy({ requireSubtype: true })` becomes the default.** Consumers explicitly opt out with `{ requireSubtype: false }` during migration.
|
||||
|
||||
7.30.1's `audit()` is the diagnostic; 8.0's `fillSubtypes()` is the bulk fixer. Together they close the migration gap deterministically.
|
||||
|
||||
## Reference
|
||||
|
||||
### Layer 1 — `subtype` (nouns)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue