David Snelling
3fffd9c6e6
On a production store (14,647 nouns / 73,070 verbs) a repairIndex() ran for more than thirty minutes at roughly a full core with ZERO log lines between its start and its end, while the read doors kept serving. The operator could tell it was alive only from `top`, and could not tell which of its single-threaded walks it was inside. Same law as the open, applied to the repair: - every phase announces itself BEFORE it works, naming what it is about to walk (each canonical walk, the VFS containment reconciliation, each provider's invariant pass); - an unref'd heartbeat names the phase still running every 5s, for as long as it runs; - every phase reports its own wall, and that wall is carried in the TYPED receipt as RepairFamilyReport.durationMs — a receipt that cannot say where the time went is not a receipt; - the whole repair's narration moves to the always-visible channel, so a production log level cannot silence it. The phases move into runRepairIndexPhases() so the heartbeat can live in a finally around them; the public door and its report shape are unchanged apart from the added durationMs. Pins: tests/integration/repair-narration.test.ts — every checked family has a start line, a finish line with its wall, and a numeric durationMs in the receipt; a phase slowed to 6.5s produces a heartbeat naming it, with the logger clamped to ERROR.
2407 lines
No EOL
96 KiB
TypeScript
2407 lines
No EOL
96 KiB
TypeScript
/**
|
||
* @module types/brainy.types
|
||
* @description Public type definitions for the Brainy API — the entity/relation
|
||
* shapes, the `find()`/`add()`/`relate()` parameter and result types, and the
|
||
* configuration surface. (Version-agnostic header: these track the package
|
||
* version, not a fixed number.)
|
||
*
|
||
* Beautiful, consistent, type-safe interfaces for the future of neural databases
|
||
*/
|
||
|
||
import { StorageAdapter, Vector } from '../coreTypes.js'
|
||
import type { EntityVisibility } from '../coreTypes.js'
|
||
import { NounType, VerbType } from './graphTypes.js'
|
||
import type {
|
||
EntityMetadataInput,
|
||
EntityMetadataPatch,
|
||
RelationMetadataInput,
|
||
RelationMetadataPatch
|
||
} from './reservedFields.js'
|
||
|
||
// ============= Core Types =============
|
||
|
||
/**
|
||
* Entity (Noun) — the fundamental data unit in Brainy
|
||
*
|
||
* **Data vs Metadata:**
|
||
* - `data`: Content used for vector embeddings. Searchable via **semantic similarity**
|
||
* (HNSW index) and hybrid text+semantic search. NOT queryable via `where` filters.
|
||
* - `metadata`: Structured fields indexed by MetadataIndex. Queryable via `where`
|
||
* filters in `find()`. Standard system fields (noun, createdAt, etc.) are stored
|
||
* alongside user metadata but extracted to top-level Entity fields on read.
|
||
*/
|
||
export interface Entity<T = any> {
|
||
/** Unique identifier — a UUID (auto-generated as a time-ordered v7; a supplied natural key is normalized to a stable v5). */
|
||
id: string
|
||
/** Embedding vector (384-dim by default). Empty array when loaded without includeVectors. */
|
||
vector: Vector
|
||
/** Entity type classification (NounType enum) */
|
||
type: NounType
|
||
/**
|
||
* Per-product sub-classification within the NounType (e.g. a `Person` entity might
|
||
* have `subtype: 'employee'` or `'customer'`; a `Document` might have `subtype: 'invoice'`).
|
||
* Flat string, no hierarchy. Top-level standard field — indexed on the fast path and
|
||
* rolled into per-NounType statistics.
|
||
*/
|
||
subtype?: string
|
||
/**
|
||
* Visibility tier (reserved top-level field). Absent === `'public'` (counted and
|
||
* returned everywhere). `'internal'` hides the entity from default `find()` / counts /
|
||
* `stats()` (opt back in with `find({ includeInternal: true })`); `'system'` is Brainy
|
||
* plumbing, hidden unless `find({ includeSystem: true })`. See {@link EntityVisibility}.
|
||
*/
|
||
visibility?: EntityVisibility
|
||
/** Opaque content — used for embeddings and semantic search. Not indexed by MetadataIndex. */
|
||
data?: any
|
||
/** User-defined structured fields — indexed and queryable via `where` filters. */
|
||
metadata?: T
|
||
/** Multi-tenancy service identifier */
|
||
service?: string
|
||
/** Creation timestamp (ms since epoch) */
|
||
createdAt: number
|
||
/** Last update timestamp (ms since epoch) */
|
||
updatedAt?: number
|
||
/** Source that created this entity (e.g., augmentation info) */
|
||
createdBy?: string
|
||
/** Type classification confidence (0-1) */
|
||
confidence?: number
|
||
/** Entity importance/salience (0-1) */
|
||
weight?: number
|
||
/**
|
||
* Monotonic revision counter, auto-bumped on every successful `update()`.
|
||
* Starts at `1` on `add()`. Pre-7.31.0 entities without `_rev` are read as `1`.
|
||
* Pass back as `update({ id, ..., ifRev: _rev })` for optimistic-concurrency CAS —
|
||
* throws `RevisionConflictError` if the persisted rev moved since read.
|
||
*/
|
||
_rev?: number
|
||
}
|
||
|
||
/**
|
||
* Relation (Verb) — a typed edge connecting two entities
|
||
*
|
||
* **Data vs Metadata (on relationships):**
|
||
* - `data`: Opaque content stored on the relationship. If provided during relate(),
|
||
* overrides the auto-computed vector (default: average of source+target vectors).
|
||
* - `metadata`: Structured queryable fields on the edge (e.g., role, startDate).
|
||
*/
|
||
export interface Relation<T = any> {
|
||
/** Unique identifier (UUID v4) */
|
||
id: string
|
||
/** Source entity ID */
|
||
from: string
|
||
/** Target entity ID */
|
||
to: string
|
||
/** Relationship type classification (VerbType enum) */
|
||
type: VerbType
|
||
/**
|
||
* Per-product sub-classification within the VerbType (e.g. a `ReportsTo`
|
||
* relationship might have `subtype: 'direct'` vs `'dotted-line'`; a `RelatedTo`
|
||
* edge might carry `'spouse'` / `'sibling'` / `'colleague'`). Flat string, no
|
||
* hierarchy. Top-level standard field — indexed on the fast path and rolled into
|
||
* per-VerbType statistics so queries like `related({ verb, subtype })` hit
|
||
* the column-store directly.
|
||
*/
|
||
subtype?: string
|
||
/**
|
||
* Visibility tier (reserved top-level field), the verb mirror of `Entity.visibility`.
|
||
* Absent === `'public'` (counted and returned everywhere). `'internal'` hides the edge
|
||
* from default `related()` / counts / `stats()` (opt back in with
|
||
* `related({ includeInternal: true })`); `'system'` is Brainy plumbing. See
|
||
* {@link EntityVisibility}.
|
||
*/
|
||
visibility?: EntityVisibility
|
||
/** Connection strength (0-1, default: 1.0) */
|
||
weight?: number
|
||
/** Opaque content for the relationship (overrides auto-computed vector if provided) */
|
||
data?: any
|
||
/** User-defined structured fields on the edge */
|
||
metadata?: T
|
||
/** Multi-tenancy service identifier */
|
||
service?: string
|
||
/** Creation timestamp (ms since epoch) */
|
||
createdAt: number
|
||
/** Last update timestamp (ms since epoch) */
|
||
updatedAt?: number
|
||
/** Relationship certainty (0-1) */
|
||
confidence?: number
|
||
/** Evidence for why this relationship was detected */
|
||
evidence?: RelationEvidence
|
||
}
|
||
|
||
/**
|
||
* Evidence for why a relationship was detected
|
||
*/
|
||
export interface RelationEvidence {
|
||
sourceText?: string // Text that indicated this relationship
|
||
position?: { // Position in source text
|
||
start: number
|
||
end: number
|
||
}
|
||
method: 'neural' | 'pattern' | 'structural' | 'explicit' // How it was detected
|
||
reasoning?: string // Human-readable explanation
|
||
}
|
||
|
||
/**
|
||
* Search result with similarity score
|
||
*
|
||
* Flattens commonly-used entity fields to the top level for convenience, while
|
||
* keeping the complete record under `entity` for callers that need the full shape.
|
||
*/
|
||
export interface Result<T = any> {
|
||
// Search metadata
|
||
id: string
|
||
score: number
|
||
|
||
// Convenience: Common entity fields flattened to top level
|
||
type?: NounType // Entity type (from entity.type)
|
||
subtype?: string // Per-product sub-classification (from entity.subtype)
|
||
visibility?: EntityVisibility // Visibility tier (from entity.visibility); absent === 'public'
|
||
metadata?: T // Entity metadata (from entity.metadata)
|
||
data?: any // Entity data (from entity.data)
|
||
confidence?: number // Type classification confidence (from entity.confidence)
|
||
weight?: number // Entity importance (from entity.weight)
|
||
_rev?: number // Monotonic revision counter (from entity._rev) — pass back as update({ ifRev }) for CAS
|
||
|
||
// Full entity record (the flattened fields above are projections of this)
|
||
entity: Entity<T>
|
||
|
||
// Score transparency
|
||
explanation?: ScoreExplanation
|
||
|
||
// Match visibility - shows what matched in hybrid search
|
||
textMatches?: string[] // Query words found in entity (e.g., ["david", "warrior"])
|
||
textScore?: number // Normalized text match score (0-1)
|
||
semanticScore?: number // Semantic similarity score (0-1)
|
||
matchSource?: 'text' | 'semantic' | 'both' // Where this result came from
|
||
rrfScore?: number // Raw RRF fusion score (for advanced ranking analysis)
|
||
|
||
// Aggregation: present only on rows returned by find({ aggregate }). These mirror the
|
||
// documented AggregateResult shape so callers can read group/metric data at the top level
|
||
// instead of digging into `metadata`. (The same values are also mirrored into `metadata`
|
||
// so aggregate-unaware callers still see them.)
|
||
groupKey?: Record<string, string | number> // Group-by key values for this aggregate row
|
||
metrics?: Record<string, number> // Computed metric values (sum/avg/min/max/count)
|
||
count?: number // Total entity count in this group
|
||
}
|
||
|
||
/**
|
||
* Score explanation for transparency
|
||
*/
|
||
export interface ScoreExplanation {
|
||
vectorScore?: number
|
||
metadataScore?: number
|
||
graphScore?: number
|
||
boosts?: Record<string, number>
|
||
penalties?: Record<string, number>
|
||
}
|
||
|
||
// ============= Operation Parameters =============
|
||
|
||
/**
|
||
* Parameters for adding entities
|
||
*
|
||
* **Data vs Metadata:**
|
||
* - `data` is embedded into a vector and searchable via semantic similarity (HNSW).
|
||
* It is NOT indexed by MetadataIndex and NOT queryable via `where` filters.
|
||
* - `metadata` is indexed by MetadataIndex and queryable via `where` filters in `find()`.
|
||
*/
|
||
/**
|
||
* Declaration-merging registry for per-`(type, subtype)` metadata shapes
|
||
* (Brainy 8.0, per BRAINY-8.0-SUBTYPE-CONTRACT § C-3).
|
||
*
|
||
* Consumers extend this interface in their own code to tell TypeScript what
|
||
* shape `metadata` takes for a given `(NounType, subtype)` pair. Brainy
|
||
* doesn't ship any entries — every entry is a consumer concern.
|
||
*
|
||
* @example
|
||
* ```ts
|
||
* declare module '@soulcraftlabs/brainy' {
|
||
* interface SubtypeRegistry {
|
||
* // For NounType.Person, subtype 'employee':
|
||
* 'person:employee': { employeeId: string; department: string }
|
||
* // For NounType.Document, subtype 'invoice':
|
||
* 'document:invoice': { invoiceNumber: string; amount: number }
|
||
* }
|
||
* }
|
||
* ```
|
||
*
|
||
* Keys are `${NounType}:${subtype}`. Values describe the metadata shape.
|
||
* Brainy reads this registry (when present) to give typed `metadata`
|
||
* autocomplete + type-checking on `add()` / `update()` / `find()`. Consumers
|
||
* with no registry entries see no type-level change — the metadata bag
|
||
* stays `T = any`.
|
||
*/
|
||
export interface SubtypeRegistry {
|
||
// Intentionally empty. Consumers extend via declaration merging.
|
||
}
|
||
|
||
/**
|
||
* A single back-fill rule for `brain.fillSubtypes()`.
|
||
*
|
||
* - A **literal string** assigns that subtype to every matching entry that
|
||
* lacks one (`'general'` — a blanket default).
|
||
* - A **function** receives the full entry and returns the subtype to assign,
|
||
* or `undefined` to leave the entry untouched (it is counted as `skipped`
|
||
* so a later run with a stricter rule can pick it up). Functions can derive
|
||
* the subtype from existing fields, e.g. `(e) => e.metadata?.kind`.
|
||
*
|
||
* @typeParam E - The entry shape the rule sees: `Entity<T>` for NounType keys,
|
||
* `Relation<T>` for VerbType keys.
|
||
*/
|
||
export type FillSubtypeRule<E> = string | ((entry: E) => string | undefined)
|
||
|
||
/**
|
||
* Rule map for `brain.fillSubtypes()` — the 8.0 subtype migration helper.
|
||
*
|
||
* Keys are `NounType` values (entity rules) and/or `VerbType` values
|
||
* (relationship rules); the two vocabularies don't overlap, so a single map
|
||
* covers both sides. Each value is a {@link FillSubtypeRule}: a literal
|
||
* subtype string or a function deriving one from the entry.
|
||
*
|
||
* @example
|
||
* ```ts
|
||
* await brain.fillSubtypes({
|
||
* [NounType.Person]: (e) => e.metadata?.kind ?? 'unspecified',
|
||
* [NounType.Thing]: 'general', // literal default
|
||
* [VerbType.RelatedTo]: 'unspecified' // relationship rule
|
||
* })
|
||
* ```
|
||
*/
|
||
export type FillSubtypeRules<T = any> = {
|
||
[K in NounType]?: FillSubtypeRule<Entity<T>>
|
||
} & {
|
||
[K in VerbType]?: FillSubtypeRule<Relation<T>>
|
||
}
|
||
|
||
/**
|
||
* Summary returned by `brain.fillSubtypes()`.
|
||
*
|
||
* After a run, `skipped` is exactly the remaining migration debt — re-running
|
||
* `brain.audit()` reports the same entries. Entries that already carry a
|
||
* subtype count toward `scanned` only (they are not debt, so they are neither
|
||
* `filled` nor `skipped`).
|
||
*/
|
||
export interface FillSubtypesResult {
|
||
/** Total entries examined (entities + relationships). */
|
||
scanned: number
|
||
/** Entries that received a subtype during this run. */
|
||
filled: number
|
||
/**
|
||
* Entries still missing a subtype after the run — either their type has no
|
||
* rule in the map, or their rule function returned `undefined`/empty.
|
||
*/
|
||
skipped: number
|
||
/** Per-entry write failures (`fillSubtypes` continues past individual errors). */
|
||
errors: Array<{ id: string; error: string }>
|
||
/** Fill counts grouped by NounType/VerbType key (only types with fills appear). */
|
||
byType: Record<string, number>
|
||
}
|
||
|
||
export interface AddParams<T = any> {
|
||
/** Content to embed and store. Strings are auto-embedded; objects are JSON-stringified for embedding. */
|
||
data: any | Vector
|
||
/** Entity type classification (required) */
|
||
type: NounType
|
||
/**
|
||
* Per-product sub-classification within the NounType (e.g. a `Person` entity might have
|
||
* `subtype: 'employee'` or `'customer'`; a `Document` might have `subtype: 'invoice'`).
|
||
* Flat string, no hierarchy — consumers choose the vocabulary. Indexed and rolled up into
|
||
* per-NounType statistics for fast `find({ type, subtype })` filtering and `groupBy:['subtype']`
|
||
* aggregation on the standard-field fast path.
|
||
*/
|
||
subtype?: string
|
||
/**
|
||
* Visibility tier (reserved top-level field). Omit (or pass `'public'`) for normal
|
||
* data that is counted and returned everywhere. Pass `'internal'` for app-internal
|
||
* data that should be hidden from default `find()` / counts / `stats()` but stay
|
||
* retrievable via `find({ includeInternal: true })`. The `'system'` tier is reserved
|
||
* for Brainy's own plumbing and is intentionally not accepted here. Stored only when
|
||
* not `'public'`.
|
||
*/
|
||
visibility?: 'public' | 'internal'
|
||
/**
|
||
* Structured queryable fields — indexed by MetadataIndex, used in `where`
|
||
* filters, `orderBy`, and aggregation.
|
||
*
|
||
* THE FIELD-ADDRESSING LAW: every name here is YOURS. There are no
|
||
* reserved metadata names — `confidence`, `type`, `id`, `level`, `data`,
|
||
* `content`, … are ordinary user fields that index, filter, sort, and
|
||
* aggregate like any other, and survive faithfully across restarts and
|
||
* rebuilds. Engine scalars are set only via their dedicated params
|
||
* (`confidence`, `weight`, `subtype`, …) and are queried explicitly as
|
||
* `system.<field>` (`where: { 'system.confidence': … }`). The ONE illegal
|
||
* spelling is a key starting `'system.'` — the engine's explicit address
|
||
* namespace cannot be forged; such a write refuses with a typed error.
|
||
*/
|
||
metadata?: EntityMetadataInput<T>
|
||
/** Custom entity ID. When omitted, a time-ordered UUID v7 is generated; a supplied natural-key string is normalized to a stable UUID v5. */
|
||
id?: string
|
||
/** Pre-computed embedding vector (skips auto-embedding when provided) */
|
||
vector?: Vector
|
||
/**
|
||
* DEFER THE EMBEDDING (MT5, the deferred-embedding worker): the write
|
||
* acknowledges at durability — data + metadata persisted, a durable
|
||
* pending-embed marker written — and the embedding + vector-index insert
|
||
* run on the engine's single-flight background worker. HONEST SEMANTICS:
|
||
* the row is findable by id/metadata/path IMMEDIATELY; vector/semantic
|
||
* search sees it when the background embed completes (eventual vector
|
||
* index — `getIndexStatus().pendingEmbeds` counts the backlog, and
|
||
* `awaitPendingEmbeds()` is the barrier). CRASH-SAFE: markers persist
|
||
* before the ack and are recovered at the next open — a crash can DELAY
|
||
* a vector, never lose one. Refused (typed) together with `vector` —
|
||
* a supplied vector has nothing to defer.
|
||
*/
|
||
deferEmbedding?: boolean
|
||
/** Multi-tenancy service identifier */
|
||
service?: string
|
||
/** Type classification confidence (0-1) */
|
||
confidence?: number
|
||
/** Entity importance/salience (0-1) */
|
||
weight?: number
|
||
/** Track which augmentation created this entity */
|
||
createdBy?: { augmentation: string; version: string }
|
||
/**
|
||
* Conditional insert. When `true` AND a custom `id` is supplied AND an entity with
|
||
* that `id` already exists, `add()` returns the existing `id` without writing — no
|
||
* throw, no overwrite. Used for idempotent bootstrap and create-or-noop patterns.
|
||
* Ignored when `id` is omitted (a freshly generated UUID can never collide).
|
||
*/
|
||
ifAbsent?: boolean
|
||
/**
|
||
* Create-or-update in a single call. When `true` AND a custom `id` is supplied AND
|
||
* an entity with that `id` already exists, `add()` applies the supplied fields as an
|
||
* UPDATE instead of the default destructive overwrite: metadata is MERGED (not
|
||
* replaced), `data` re-embeds only when it changed, `_rev` is bumped, and the
|
||
* original `createdAt` is PRESERVED. When no entity with that `id` exists, `add()`
|
||
* inserts normally (a fresh `_rev` of 1). Ignored — behaves as a plain insert — when
|
||
* `id` is omitted, since a freshly generated UUID can never collide.
|
||
*
|
||
* Mutually exclusive with {@link AddParams.ifAbsent}: `ifAbsent` SKIPS an existing
|
||
* entity, `upsert` MERGES into it — supplying both throws.
|
||
*
|
||
* @example
|
||
* // First call inserts; a later call with the same id merges + bumps _rev.
|
||
* await brain.add({ id: 'customer-42', type: 'customer', data: 'Acme', upsert: true })
|
||
* await brain.add({ id: 'customer-42', type: 'customer', metadata: { tier: 'gold' }, upsert: true })
|
||
*/
|
||
upsert?: boolean
|
||
}
|
||
|
||
/**
|
||
* Parameters for updating entities
|
||
*/
|
||
export interface UpdateParams<T = any> {
|
||
id: string // Entity to update
|
||
data?: any // New content to re-embed
|
||
/**
|
||
* Defer the re-embedding of new `data` (see `AddParams.deferEmbedding`).
|
||
* The write acks at durability; the OLD vector keeps serving semantic
|
||
* search — stale-but-present, never absent (the flicker law) — until the
|
||
* background worker embeds the new content and swaps it in atomically.
|
||
* `data` reads return the NEW content immediately. Refused (typed) with
|
||
* an explicit `vector`.
|
||
*/
|
||
deferEmbedding?: boolean
|
||
type?: NounType // Change type
|
||
subtype?: string // Change subtype (set to '' or null-equivalent via dedicated unset is future work)
|
||
/**
|
||
* Change the visibility tier. Omit to preserve the existing value. Toggling between
|
||
* `'public'` and `'internal'` moves the entity in/out of the default-visible counts and
|
||
* `find()` results. The `'system'` tier is Brainy-internal (e.g. re-asserted on the VFS
|
||
* root during repair) — consumer code should only ever use `'public'` / `'internal'`.
|
||
*/
|
||
visibility?: EntityVisibility
|
||
/**
|
||
* Metadata fields to merge (or replace when `merge: false`). Every name is
|
||
* the user's (the field-addressing law) — a patch field named `confidence`
|
||
* updates YOUR field of that name, never the engine scalar (use the
|
||
* dedicated `confidence` param for that). Keys spelled `'system.…'` refuse
|
||
* with a typed error (namespace forgery).
|
||
*/
|
||
metadata?: EntityMetadataPatch<T>
|
||
merge?: boolean // Merge or replace metadata (default: true)
|
||
vector?: Vector // New pre-computed vector
|
||
confidence?: number // Update type classification confidence
|
||
weight?: number // Update entity importance/salience
|
||
/**
|
||
* Optimistic concurrency check. When provided, the update fails with
|
||
* `RevisionConflictError` if the persisted entity's `_rev` does not equal `ifRev`.
|
||
* `_rev` is auto-bumped on every successful update — read it from the entity returned
|
||
* by `get()` / `find()` / `search()`. Omit to skip the check (unconditional update).
|
||
*/
|
||
ifRev?: number
|
||
}
|
||
|
||
/**
|
||
* Parameters for creating relationships
|
||
*
|
||
* **Data vs Metadata (on relationships):**
|
||
* - `data`: Opaque content for the edge. If provided, overrides the auto-computed
|
||
* vector (default: average of source+target entity vectors).
|
||
* - `metadata`: Structured queryable fields on the edge.
|
||
*/
|
||
export interface RelateParams<T = any> {
|
||
/** Source entity ID (required — must exist) */
|
||
from: string
|
||
/** Target entity ID (required — must exist) */
|
||
to: string
|
||
/** Relationship type classification (required) */
|
||
type: VerbType
|
||
/**
|
||
* Per-product sub-classification within the VerbType (e.g. a `ReportsTo`
|
||
* relationship might have `subtype: 'direct'` or `'dotted-line'`; a `RelatedTo`
|
||
* edge might carry `'spouse'` or `'colleague'`). Flat string, no hierarchy.
|
||
* Indexed and rolled up into per-VerbType statistics for fast filtering
|
||
* (`related({ verb, subtype })`) and aggregation (`groupBy:['subtype']`).
|
||
*/
|
||
subtype?: string
|
||
/**
|
||
* Visibility tier (reserved top-level field), the verb mirror of `AddParams.visibility`.
|
||
* Omit (or pass `'public'`) for normal edges that are counted and returned everywhere.
|
||
* Pass `'internal'` for app-internal edges hidden from default `related()` / counts /
|
||
* `stats()` but retrievable via `related({ includeInternal: true })`. The `'system'`
|
||
* tier is reserved for Brainy's own plumbing and is not accepted here. Stored only when
|
||
* not `'public'`.
|
||
*/
|
||
visibility?: 'public' | 'internal'
|
||
/** Connection strength (0-1, default: 1.0) */
|
||
weight?: number
|
||
/** Content for the relationship (optional — overrides auto-computed vector) */
|
||
data?: any
|
||
/**
|
||
* Structured queryable fields on the edge. Every name is the user's (the
|
||
* field-addressing law) — `verb`, `confidence`, `weight`, … in this bag are
|
||
* ordinary user fields; engine scalars ride their dedicated params and are
|
||
* addressed as `system.<field>`. Keys spelled `'system.…'` refuse with a
|
||
* typed error (namespace forgery).
|
||
*/
|
||
metadata?: RelationMetadataInput<T>
|
||
/** Create reverse edge too (default: false) */
|
||
bidirectional?: boolean
|
||
/** Multi-tenancy service identifier */
|
||
service?: string
|
||
/** Relationship certainty (0-1) */
|
||
confidence?: number
|
||
/** Evidence for why this relationship exists */
|
||
evidence?: RelationEvidence
|
||
}
|
||
|
||
/**
|
||
* Parameters for updating relationships
|
||
*/
|
||
export interface UpdateRelationParams<T = any> {
|
||
id: string // Relation to update
|
||
type?: VerbType // Change verb type
|
||
subtype?: string // Change sub-classification (omit to preserve existing)
|
||
/**
|
||
* Change the visibility tier. Omit to preserve the existing value. The verb mirror of
|
||
* `UpdateParams.visibility`; `'system'` is Brainy-internal — consumer code should only
|
||
* use `'public'` / `'internal'`.
|
||
*/
|
||
visibility?: EntityVisibility
|
||
weight?: number // New weight
|
||
confidence?: number // New confidence (0-1)
|
||
data?: any // New content
|
||
/**
|
||
* Metadata fields to merge (or replace when `merge: false`). Every name is
|
||
* the user's (the field-addressing law); engine scalars ride their
|
||
* dedicated params. Keys spelled `'system.…'` refuse with a typed error.
|
||
*/
|
||
metadata?: RelationMetadataPatch<T>
|
||
merge?: boolean // Merge or replace metadata
|
||
}
|
||
|
||
// ============= Query Parameters =============
|
||
|
||
/**
|
||
* Unified find parameters — Triple Intelligence search
|
||
*
|
||
* Combines three search dimensions in one query:
|
||
* - **Vector:** `query` or `vector` for semantic/hybrid similarity search (searches `data`)
|
||
* - **Metadata:** `where` for structured field filters (queries `metadata` via MetadataIndex)
|
||
* - **Graph:** `connected` for relationship traversal (via GraphAdjacencyIndex)
|
||
*
|
||
* See also: [Query Operators](../../docs/QUERY_OPERATORS.md) for all `where` operators.
|
||
*
|
||
* @remarks
|
||
* **Field-addressing law.** Governs every query-surface field name — `where`
|
||
* and `orderBy` on this interface, plus `AggregateSource.where` and
|
||
* `AggregateDefinition.groupBy` in the aggregation engine:
|
||
*
|
||
* 1. A bare name (e.g. `'level'`, `'rank'`, `'score'`) always means the
|
||
* caller's own metadata field — it reads `entity.metadata.<name>`. There
|
||
* is no fallback to an engine-internal field of the same name and no
|
||
* priority resolution between the two; metadata wins unconditionally.
|
||
* 2. `system.<field>` reaches an engine scalar, explicitly, and only for
|
||
* these ten: `id`, `type`, `subtype`, `createdAt`, `updatedAt`,
|
||
* `confidence`, `weight`, `visibility`, `service`, `createdBy`.
|
||
* 3. `vector`, `connections`, `level` (the engine-internal node field — a
|
||
* different thing from a user metadata field also named `level`),
|
||
* `data`, and `_rev` are invisible plumbing: neither spelling can
|
||
* address them from a query surface.
|
||
* 4. `metadata.<field>` is the explicit spelling of the bare form and means
|
||
* exactly the same thing as rule 1.
|
||
* 5. A name that matches none of the above — most often a bare name that
|
||
* collides with one of the ten system-scalar names in rule 2 — REFUSES
|
||
* with a typed {@link UnresolvableFieldError} naming both candidates,
|
||
* e.g. `no metadata field 'createdAt' — did you mean system.createdAt or
|
||
* metadata.createdAt?`. The same loud-refusal principle covers whole
|
||
* options: the previously accepted-and-silently-ignored `cursor`,
|
||
* `includeRelations`, and `writeOnly` now throw
|
||
* {@link UnsupportedFindOptionError} instead of doing nothing.
|
||
* 6. **Ordering contract** (identical on the pure-JS engine and the native
|
||
* accelerator): rows missing or `null` on the `orderBy` field sort LAST
|
||
* in BOTH `asc` and `desc` order and are never dropped from the result;
|
||
* ties break by `id` ascending.
|
||
*
|
||
* Migration note: a call site written against the old rule — e.g.
|
||
* `orderBy: 'createdAt'` or `where: { visibility: 'internal' }` meaning the
|
||
* engine scalar — now refuses instead of silently reading the wrong field.
|
||
* The thrown error names the exact fix (`system.createdAt`). A loud
|
||
* refusal with the fix in hand beats a silent behavior flip.
|
||
*/
|
||
export interface FindParams<T = any> {
|
||
// Vector Intelligence
|
||
/** Natural language or semantic search query (embedded and matched via HNSW + text index) */
|
||
query?: string
|
||
/** Direct vector search (pre-computed embedding) */
|
||
vector?: Vector
|
||
|
||
// Metadata Intelligence
|
||
/** Filter by entity type(s). Alias for `where.noun`. */
|
||
type?: NounType | NounType[]
|
||
/**
|
||
* Filter by per-product subtype (top-level standard field — uses the fast path, not the
|
||
* metadata fallback). Pass a single string for equality, or an array for set membership
|
||
* (e.g. `subtype: ['employee', 'contractor']`). For operator-form predicates (e.g.
|
||
* `{ exists: true }`, `{ missing: true }`) use `where: { subtype: { …operators… } }`.
|
||
*/
|
||
subtype?: string | string[]
|
||
/**
|
||
* Metadata filters using BFO operators (e.g., `{ year: { greaterThan: 2020 } }`).
|
||
* Field names follow the field-addressing law — see the `@remarks` on
|
||
* {@link FindParams}: a bare key is always the caller's metadata field;
|
||
* an engine scalar needs the explicit `system.<field>` form.
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* await brain.find({ where: { level: { greaterThan: 5 } } }) // metadata.level
|
||
* await brain.find({ where: { 'system.visibility': 'internal' } }) // engine scalar
|
||
* ```
|
||
*/
|
||
where?: Partial<T>
|
||
|
||
// Visibility
|
||
/**
|
||
* Also return `visibility: 'internal'` entities. By default `find()` returns ONLY
|
||
* public entities (those with no `visibility` field, or `visibility: 'public'`);
|
||
* app-internal entities are hidden. Set `true` to include them. Applied as a hard
|
||
* candidate filter, so `limit` / `offset` stay correct. Does NOT include `'system'`
|
||
* entities — use `includeSystem` for those.
|
||
*/
|
||
includeInternal?: boolean
|
||
/**
|
||
* Also return `visibility: 'system'` entities (Brainy's own plumbing, e.g. the VFS
|
||
* root). Hidden by default and even when `includeInternal` is set. Set `true` only
|
||
* when you specifically need to see system entities. Applied as a hard candidate
|
||
* filter, so `limit` / `offset` stay correct.
|
||
*/
|
||
includeSystem?: boolean
|
||
|
||
// Graph Intelligence
|
||
connected?: GraphConstraints
|
||
|
||
// Proximity search
|
||
near?: {
|
||
id: string // Find near this entity
|
||
threshold?: number // Min similarity (0-1)
|
||
}
|
||
|
||
// Control options
|
||
limit?: number // Max results (default: 10)
|
||
offset?: number // Skip N results
|
||
/**
|
||
* @deprecated Not implemented. Passing `cursor` throws
|
||
* {@link UnsupportedFindOptionError} — it used to be accepted and
|
||
* silently ignored, which masked that no cursor pagination ever ran. Use
|
||
* `offset` / `limit` until cursor pagination ships.
|
||
*/
|
||
cursor?: string // Cursor-based pagination
|
||
|
||
// Sorting
|
||
/**
|
||
* Field to sort by. Follows the field-addressing law (see the `@remarks`
|
||
* on {@link FindParams}): a bare name (`'level'`, `'rank'`, `'score'`, …)
|
||
* always sorts by that metadata field; the ten engine scalars sort only
|
||
* via the explicit `system.<field>` form (e.g. `'system.createdAt'`); a
|
||
* name that resolves to neither throws {@link UnresolvableFieldError}
|
||
* naming the fix.
|
||
*
|
||
* Ordering contract (identical on the pure-JS engine and the native
|
||
* accelerator): rows missing or `null` on this field sort LAST in BOTH
|
||
* `asc` and `desc` order and are never dropped from the result; ties
|
||
* break by `id` ascending.
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* await brain.find({ orderBy: 'level', order: 'desc' }) // metadata.level
|
||
* await brain.find({ orderBy: 'system.createdAt', order: 'desc' }) // engine scalar
|
||
* ```
|
||
*/
|
||
orderBy?: string
|
||
/**
|
||
* Sort direction: `'asc'` (default) or `'desc'`. Per the ordering
|
||
* contract on `orderBy`, rows missing/`null` on the sorted field sort
|
||
* LAST in both directions — `order` never moves them to the front.
|
||
*/
|
||
order?: 'asc' | 'desc' // Sort direction: 'asc' (default) or 'desc'
|
||
|
||
// Advanced options
|
||
/**
|
||
* @deprecated Not implemented. Passing `includeRelations` throws
|
||
* {@link UnsupportedFindOptionError} — it used to be accepted and
|
||
* silently ignored, so no relationships were ever attached. Fetch
|
||
* relationships separately via `brain.related()`.
|
||
*/
|
||
includeRelations?: boolean // Include entity relationships
|
||
excludeVFS?: boolean // Exclude VFS entities from results (default: false - VFS included)
|
||
service?: string // Multi-tenancy filter
|
||
/**
|
||
* Return each result's stored embedding under `entity.vector`. Defaults to `false`,
|
||
* in which case `entity.vector` is an empty array (the perf default — vectors are
|
||
* large and most reads don't need them). Set `true` to get the persisted vector
|
||
* back from `find()` without a recompute, e.g. to feed downstream similarity math.
|
||
* Mirrors {@link GetOptions.includeVectors} on `get()`.
|
||
*/
|
||
includeVectors?: boolean
|
||
|
||
// Hybrid search options
|
||
/**
|
||
* Search strategy — the single canonical search-mode knob (the redundant `mode`
|
||
* alias was removed in 8.0). `'auto'` (default) blends text + semantic; the
|
||
* other members force a single intelligence. See {@link SearchMode}.
|
||
*/
|
||
searchMode?: SearchMode
|
||
hybridAlpha?: number // Weight between text (0.0) and semantic (1.0) search, default: auto-detected by query length
|
||
|
||
// Triple Intelligence Fusion
|
||
fusion?: {
|
||
strategy?: 'adaptive' | 'weighted' | 'progressive'
|
||
weights?: {
|
||
vector?: number
|
||
graph?: number
|
||
field?: number
|
||
}
|
||
}
|
||
|
||
// Performance options
|
||
/**
|
||
* @deprecated Not implemented. Passing `writeOnly` throws
|
||
* {@link UnsupportedFindOptionError} — it used to be accepted and
|
||
* silently ignored, so validation was never actually skipped.
|
||
*/
|
||
writeOnly?: boolean // Skip validation for high-speed ingestion
|
||
|
||
// Aggregation
|
||
/** Query a named aggregate definition. String shorthand or full query params. */
|
||
aggregate?: string | AggregateQueryParams
|
||
}
|
||
|
||
/**
|
||
* Graph constraints for search
|
||
*/
|
||
export interface GraphConstraints {
|
||
to?: string // Connected to this entity
|
||
from?: string // Connected from this entity
|
||
via?: VerbType | VerbType[] // Traverse via these relationship types (the canonical key)
|
||
type?: VerbType | VerbType[] // Accepted alias for `via` (resolved identically: `via ?? type`)
|
||
/**
|
||
* Filter traversal edges by VerbType subtype. Single string for equality,
|
||
* array for set membership. Composes with `via` — `{ via: 'manages', subtype:
|
||
* 'direct' }` traverses only direct-management edges, not dotted-line. Edges
|
||
* without a subtype value are excluded when this filter is set.
|
||
*/
|
||
subtype?: string | string[]
|
||
depth?: number // Max traversal depth (default: 1)
|
||
direction?: 'in' | 'out' | 'both' // Direction of traversal (default: 'both')
|
||
}
|
||
|
||
/**
|
||
* Search modes
|
||
*/
|
||
export type SearchMode =
|
||
| 'auto' // Automatically choose best mode (default - enables hybrid text+semantic)
|
||
| 'vector' // Pure vector search (semantic only)
|
||
| 'semantic' // Alias for vector search
|
||
| 'text' // Pure text/keyword search
|
||
| 'metadata' // Pure metadata filtering
|
||
| 'graph' // Pure graph traversal
|
||
| 'hybrid' // Combine all intelligences (explicit hybrid mode)
|
||
|
||
/**
|
||
* Parameters for similarity search
|
||
*/
|
||
export interface SimilarParams<T = any> {
|
||
to: string | Entity<T> | Vector // Find similar to this
|
||
limit?: number // Max results (default: 10)
|
||
threshold?: number // Min similarity score
|
||
type?: NounType | NounType[] // Restrict to types
|
||
where?: Partial<T> // Additional filters
|
||
service?: string // Multi-tenancy
|
||
excludeVFS?: boolean // Exclude VFS entities (default: false - VFS included)
|
||
}
|
||
|
||
/**
|
||
* Parameters for `brain.related()` / `db.related()`
|
||
*
|
||
* All parameters are optional. When called without parameters, returns all relationships
|
||
* with pagination (default limit: 100).
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* // Get all relationships (default limit: 100)
|
||
* const all = await brain.related()
|
||
*
|
||
* // Get relationships from a specific entity (string shorthand)
|
||
* const fromEntity = await brain.related(entityId)
|
||
*
|
||
* // Equivalent to:
|
||
* const fromEntity2 = await brain.related({ from: entityId })
|
||
*
|
||
* // Get relationships to a specific entity
|
||
* const toEntity = await brain.related({ to: entityId })
|
||
*
|
||
* // Filter by relationship type
|
||
* const friends = await brain.related({ type: VerbType.FriendOf })
|
||
*
|
||
* // Pagination
|
||
* const page2 = await brain.related({ offset: 100, limit: 50 })
|
||
*
|
||
* // Combined filters
|
||
* const filtered = await brain.related({
|
||
* from: entityId,
|
||
* type: VerbType.WorksWith,
|
||
* limit: 20
|
||
* })
|
||
* ```
|
||
*
|
||
* Fixed bug where calling without parameters returned empty array
|
||
* Added string ID shorthand syntax
|
||
*/
|
||
export interface RelatedParams {
|
||
/**
|
||
* Filter by source entity ID
|
||
*
|
||
* Returns all relationships originating from this entity.
|
||
*/
|
||
from?: string
|
||
|
||
/**
|
||
* Filter by target entity ID
|
||
*
|
||
* Returns all relationships pointing to this entity.
|
||
*/
|
||
to?: string
|
||
|
||
/**
|
||
* Filter by INCIDENT entity ID — every relationship touching this entity in
|
||
* EITHER direction (where it is the source OR the target), deduped. The
|
||
* one-call "all edges on a noun" query: equivalent to merging
|
||
* `related({ from: id })` and `related({ to: id })` but in a single
|
||
* O(degree) call. Mutually exclusive with `from` / `to` (pass `node` for
|
||
* both directions, or `from` / `to` for one). Composes with `type` /
|
||
* `subtype` / visibility filters.
|
||
*
|
||
* @example
|
||
* // Everything connected to this person, in or out.
|
||
* const edges = await brain.related({ node: personId })
|
||
*/
|
||
node?: string
|
||
|
||
/**
|
||
* Filter by relationship type(s)
|
||
*
|
||
* Can be a single VerbType or array of VerbTypes.
|
||
*/
|
||
type?: VerbType | VerbType[]
|
||
|
||
/**
|
||
* Filter by VerbType subtype
|
||
*
|
||
* Top-level standard field — uses the fast path, not the metadata fallback.
|
||
* Pass a single string for equality (e.g. `subtype: 'direct'`), or an array
|
||
* for set membership (e.g. `subtype: ['direct', 'dotted-line']`).
|
||
*/
|
||
subtype?: string | string[]
|
||
|
||
/**
|
||
* Also return `visibility: 'internal'` relationships.
|
||
*
|
||
* By default `related()` returns ONLY public relationships (those with no
|
||
* `visibility` field, or `visibility: 'public'`); app-internal edges are hidden.
|
||
* Set `true` to include them. Applied as a hard candidate filter so `limit` /
|
||
* `offset` stay correct. Does NOT include `'system'` edges — use `includeSystem`.
|
||
*/
|
||
includeInternal?: boolean
|
||
|
||
/**
|
||
* Also return `visibility: 'system'` relationships (Brainy's own plumbing).
|
||
*
|
||
* Hidden by default and even when `includeInternal` is set. Set `true` only when
|
||
* you specifically need system edges. Applied as a hard candidate filter so
|
||
* `limit` / `offset` stay correct.
|
||
*/
|
||
includeSystem?: boolean
|
||
|
||
/**
|
||
* Maximum number of results to return
|
||
*
|
||
* @default 100
|
||
*/
|
||
limit?: number
|
||
|
||
/**
|
||
* Number of results to skip (offset-based pagination)
|
||
*
|
||
* @default 0
|
||
*/
|
||
offset?: number
|
||
|
||
/**
|
||
* Cursor for cursor-based pagination
|
||
*
|
||
* More efficient than offset for large result sets.
|
||
*/
|
||
cursor?: string
|
||
|
||
/**
|
||
* Filter by service (multi-tenancy)
|
||
*
|
||
* Only return relationships belonging to this service.
|
||
*/
|
||
service?: string
|
||
}
|
||
|
||
// ============= Graph views (brain.graph.*) =============
|
||
|
||
/**
|
||
* A node in a {@link GraphView} — a lightweight reference (id + classification +
|
||
* hop depth), NOT a full {@link Entity}. Graph reads are structure-first; resolve
|
||
* a node to its full entity with `get(id)` when you need its `data` / metadata.
|
||
*/
|
||
export interface GraphNode {
|
||
/** Entity id. */
|
||
id: string
|
||
/** Entity type (present when the view hydrates node classification). */
|
||
type?: NounType
|
||
/** Per-product subtype (present when hydrated). */
|
||
subtype?: string
|
||
/** Hop distance from the nearest seed (0 = a seed); present on subgraph results. */
|
||
depth?: number
|
||
}
|
||
|
||
/**
|
||
* A hydrated view of a region of the graph — the discovered nodes plus the edges
|
||
* among them, in id form (the public face of the provider's columnar int form).
|
||
*/
|
||
export interface GraphView<T = any> {
|
||
/** The nodes in this region (seeds + everything reached). */
|
||
nodes: GraphNode[]
|
||
/** The edges among `nodes`, as full {@link Relation} objects. */
|
||
edges: Relation<T>[]
|
||
/** `true` when a `maxNodes` / `maxEdges` cap truncated the result. */
|
||
truncated?: boolean
|
||
}
|
||
|
||
/**
|
||
* Options for {@link GraphApi.subgraph}.
|
||
*/
|
||
export interface SubgraphOptions {
|
||
/** Max hop distance from any seed (default `1`). */
|
||
depth?: number
|
||
/** Edge direction to follow (default `'both'`). */
|
||
direction?: 'in' | 'out' | 'both'
|
||
/** Restrict traversed edges to these relationship type(s). */
|
||
type?: VerbType | VerbType[]
|
||
/** Restrict traversed edges to these subtype(s). */
|
||
subtype?: string | string[]
|
||
/** Also traverse through `internal`-visibility edges/nodes (hidden by default). */
|
||
includeInternal?: boolean
|
||
/** Also traverse through `system`-visibility edges/nodes (hidden by default). */
|
||
includeSystem?: boolean
|
||
/** Cap on returned nodes (sets `GraphView.truncated`). */
|
||
maxNodes?: number
|
||
/** Cap on returned edges. */
|
||
maxEdges?: number
|
||
/**
|
||
* Hydrate each node's `type` / `subtype` (one batch read). Default `true`;
|
||
* set `false` to skip it when you only need graph structure (ids + edges).
|
||
*/
|
||
hydrateNodes?: boolean
|
||
}
|
||
|
||
/**
|
||
* Options for {@link GraphApi.export}.
|
||
*/
|
||
export interface GraphExportOptions {
|
||
/** Nodes/edges per streamed chunk (default `1000`). */
|
||
chunkSize?: number
|
||
/** Also include `internal`-visibility nodes/edges (hidden by default). */
|
||
includeInternal?: boolean
|
||
/** Also include `system`-visibility nodes/edges (hidden by default). */
|
||
includeSystem?: boolean
|
||
/** Stream the nodes (default `true`; set `false` to stream only edges). */
|
||
includeNodes?: boolean
|
||
/** Stream the edges (default `true`; set `false` to stream only nodes). */
|
||
includeEdges?: boolean
|
||
}
|
||
|
||
/**
|
||
* Options for {@link GraphApi.communities}.
|
||
*/
|
||
export interface GraphCommunitiesOptions {
|
||
/** Treat edges as directed when grouping (default `false` — direction-agnostic). */
|
||
directed?: boolean
|
||
/** Also group through `internal`-visibility nodes/edges (hidden by default). */
|
||
includeInternal?: boolean
|
||
/** Also group through `system`-visibility nodes/edges (hidden by default). */
|
||
includeSystem?: boolean
|
||
}
|
||
|
||
/**
|
||
* Result of {@link GraphApi.communities} — the graph partitioned into connected
|
||
* groups of entity ids.
|
||
*/
|
||
export interface GraphCommunitiesResult {
|
||
/** Each group is the member entity ids; largest group first. */
|
||
groups: string[][]
|
||
/** Number of distinct groups (`= groups.length`). */
|
||
count: number
|
||
}
|
||
|
||
/**
|
||
* Options for {@link GraphApi.rank} (importance ranking). INTENT-level only — the
|
||
* ranking ALGORITHM is the provider's choice (the TS fallback uses PageRank), so
|
||
* no algorithm-tuning knobs are exposed.
|
||
*/
|
||
export interface GraphRankOptions {
|
||
/** Return only the top-K nodes by score (default: all, descending). */
|
||
topK?: number
|
||
/** Also rank through `internal`-visibility nodes/edges (hidden by default). */
|
||
includeInternal?: boolean
|
||
/** Also rank through `system`-visibility nodes/edges (hidden by default). */
|
||
includeSystem?: boolean
|
||
}
|
||
|
||
/** One ranked entity from {@link GraphApi.rank}, descending by `score`. */
|
||
export interface GraphRankEntry {
|
||
/** The entity id. */
|
||
id: string
|
||
/** The importance score (relative; higher = more central). */
|
||
score: number
|
||
}
|
||
|
||
/**
|
||
* Options for {@link GraphApi.path} (best route between two entities).
|
||
*/
|
||
export interface GraphPathOptions {
|
||
/** Edge direction to follow (default `'both'`). */
|
||
direction?: 'in' | 'out' | 'both'
|
||
/** Optimize for fewest `'hops'` (default) or least summed edge `'weight'`. */
|
||
by?: 'hops' | 'weight'
|
||
/** Restrict the route to these relationship type(s). */
|
||
type?: VerbType | VerbType[]
|
||
/** Also route through `internal`-visibility nodes/edges (hidden by default). */
|
||
includeInternal?: boolean
|
||
/** Also route through `system`-visibility nodes/edges (hidden by default). */
|
||
includeSystem?: boolean
|
||
/** Abandon the search past this many hops. */
|
||
maxDepth?: number
|
||
}
|
||
|
||
/**
|
||
* Result of {@link GraphApi.path} — the route from `from` to `to`, or `null` when
|
||
* unreachable.
|
||
*/
|
||
export interface GraphPathResult {
|
||
/** The entity ids along the route, `from` … `to` inclusive. */
|
||
nodes: string[]
|
||
/** The relationship (verb) ids traversed, one fewer than `nodes`. */
|
||
relationships: string[]
|
||
/** Route cost: number of hops, or summed edge weight when `by: 'weight'`. */
|
||
cost: number
|
||
}
|
||
|
||
/**
|
||
* What {@link GraphApi.subgraph} expands from — "the things to grow a neighborhood
|
||
* around". One of:
|
||
* - an entity id, or an array of them (the explicit id set),
|
||
* - a {@link Result} array (a `find()` result — its entities become the seeds), or
|
||
* - a {@link FindParams} query (the query→expand fusion: run the query, expand from
|
||
* every match). With the native engine a metadata-only query's matched universe is
|
||
* forwarded to the traversal as an opaque set with no id materialization.
|
||
*/
|
||
export type SubgraphSelector<T = any> = string | string[] | Result<T>[] | FindParams<T>
|
||
|
||
/**
|
||
* The `brain.graph` namespace — graph-shaped reads over the knowledge graph.
|
||
* Routes to a native {@link import('../plugin.js').GraphAccelerationProvider}
|
||
* when one is registered, otherwise serves the same results from Brainy's
|
||
* pure-TS adjacency (correct at small/medium scale).
|
||
*/
|
||
export interface GraphApi<T = any> {
|
||
/**
|
||
* @description Extract the subgraph reachable from a seed selector — the bounded
|
||
* multi-hop neighborhood, as nodes + edges. The one-call answer to "show me
|
||
* everything around this, N hops out". The seed can be entity id(s), a `find()`
|
||
* result, or a {@link FindParams} query (query→expand: run the query, then expand
|
||
* from every match — with the native engine the matched universe never leaves
|
||
* the engine's representation).
|
||
* @param selector - Seed id(s), a `find()` result, or a query. See {@link SubgraphSelector}.
|
||
* @param options - Depth, direction, edge/visibility filters, and caps.
|
||
* @returns The hydrated {@link GraphView}.
|
||
* @example
|
||
* const view = await brain.graph.subgraph(personId, { depth: 2 })
|
||
* // view.nodes = the 2-hop neighborhood; view.edges = the relations among them
|
||
* @example
|
||
* // Query→expand: the neighborhood of everything matching a filter.
|
||
* const cluster = await brain.graph.subgraph({ where: { team: 'platform' } }, { depth: 1 })
|
||
*/
|
||
subgraph(selector: SubgraphSelector<T>, options?: SubgraphOptions): Promise<GraphView<T>>
|
||
|
||
/**
|
||
* @description Stream the WHOLE graph in one O(N+E) pass as a sequence of
|
||
* {@link GraphView} chunks — the right primitive for visualizing all data
|
||
* (vs. paging per node). Node chunks come first, then edge chunks; assemble
|
||
* them on the consumer side. Backed by cursor pagination, so a full walk is
|
||
* O(N+E) at any chunk size (no re-scan).
|
||
* @param options - Chunk size, visibility, and node/edge inclusion toggles.
|
||
* @returns An async-iterable of graph chunks.
|
||
* @example
|
||
* for await (const chunk of brain.graph.export()) {
|
||
* addNodes(chunk.nodes); addEdges(chunk.edges)
|
||
* }
|
||
*/
|
||
export(options?: GraphExportOptions): AsyncIterable<GraphView<T>>
|
||
|
||
/**
|
||
* @description Rank entities by graph importance (influence / centrality) — the
|
||
* one-call answer to "which nodes matter most". The TS fallback uses PageRank;
|
||
* a native provider may use personalized PageRank / eigenvector centrality.
|
||
* @param options - `topK`, visibility opt-ins.
|
||
* @returns Entities with scores, DESCENDING (most important first).
|
||
* @example
|
||
* const top = await brain.graph.rank({ topK: 10 })
|
||
* top[0] // { id, score } — the most central entity
|
||
*/
|
||
rank(options?: GraphRankOptions): Promise<GraphRankEntry[]>
|
||
|
||
/**
|
||
* @description Partition the graph into connected communities (clusters of
|
||
* related entities) — "which things group together".
|
||
* @param options - `directed`, visibility opt-ins.
|
||
* @returns The groups (member ids) + their count.
|
||
* @example
|
||
* const { groups, count } = await brain.graph.communities()
|
||
*/
|
||
communities(options?: GraphCommunitiesOptions): Promise<GraphCommunitiesResult>
|
||
|
||
/**
|
||
* @description Find the best route between two entities — fewest hops (default)
|
||
* or least summed edge weight (`by: 'weight'`). The pathfinding ALGORITHM is
|
||
* the provider's choice (TS fallback: BFS for hops, Dijkstra for weight).
|
||
* @param from - Start entity id (natural keys are resolved).
|
||
* @param to - End entity id.
|
||
* @param options - Direction, `by`, type filter, `maxDepth`, visibility opt-ins.
|
||
* @returns The route (`nodes` + `relationships` + `cost`), or `null` if unreachable.
|
||
* @example
|
||
* const route = await brain.graph.path(aliceId, bobId)
|
||
* route?.nodes // [aliceId, …, bobId]
|
||
*/
|
||
path(from: string, to: string, options?: GraphPathOptions): Promise<GraphPathResult | null>
|
||
}
|
||
|
||
// ============= Batch Operations =============
|
||
|
||
/**
|
||
* Batch add parameters
|
||
*/
|
||
export interface AddManyParams<T = any> {
|
||
items: AddParams<T>[] // Items to add
|
||
parallel?: boolean // Process in parallel (default: true)
|
||
chunkSize?: number // Batch size (default: 100)
|
||
onProgress?: (done: number, total: number) => void
|
||
continueOnError?: boolean // Continue if some fail
|
||
/**
|
||
* Conditional insert applied to every item. Equivalent to setting `ifAbsent: true`
|
||
* on each item individually. Item-level `ifAbsent` overrides the batch flag.
|
||
*/
|
||
ifAbsent?: boolean
|
||
/**
|
||
* Create-or-update applied to every item. Equivalent to setting `upsert: true` on
|
||
* each item individually (see {@link AddParams.upsert}): an item whose custom `id`
|
||
* already exists is MERGED as an update rather than overwritten. Item-level `upsert`
|
||
* overrides the batch flag. Mutually exclusive with {@link AddManyParams.ifAbsent}
|
||
* at the item level — an item resolving to both throws.
|
||
*/
|
||
upsert?: boolean
|
||
}
|
||
|
||
/**
|
||
* Batch update parameters
|
||
*/
|
||
export interface UpdateManyParams<T = any> {
|
||
items: UpdateParams<T>[] // Items to update
|
||
parallel?: boolean
|
||
chunkSize?: number
|
||
onProgress?: (done: number, total: number) => void
|
||
continueOnError?: boolean
|
||
}
|
||
|
||
/**
|
||
* Batch remove parameters
|
||
*/
|
||
export interface RemoveManyParams {
|
||
ids?: string[] // Specific IDs to remove
|
||
type?: NounType // Remove all of type
|
||
where?: any // Remove by metadata
|
||
limit?: number // Max to remove (safety)
|
||
/**
|
||
* Entities removed per transaction chunk. Each chunk is one atomic transaction —
|
||
* a chunk commits or rolls back together, and failures are isolated to their chunk.
|
||
* Defaults to the storage adapter's `maxBatchSize` (memory: 1000, S3/R2: 100,
|
||
* GCS: 50), so zero-config removals match the adapter's characteristics. Override
|
||
* only to tune throughput vs. transaction granularity.
|
||
*/
|
||
chunkSize?: number
|
||
onProgress?: (done: number, total: number) => void
|
||
continueOnError?: boolean // Continue processing if a removal fails
|
||
}
|
||
|
||
/**
|
||
* Batch relate parameters
|
||
*/
|
||
export interface RelateManyParams<T = any> {
|
||
items: RelateParams<T>[] // Relations to create
|
||
parallel?: boolean
|
||
chunkSize?: number
|
||
onProgress?: (done: number, total: number) => void
|
||
continueOnError?: boolean
|
||
}
|
||
|
||
/**
|
||
* Batch result
|
||
*/
|
||
/**
|
||
* One family's row in a {@link RepairReport} — what repairIndex() checked,
|
||
* what it healed, and why anything was skipped. The receipts venue's graph
|
||
* trust program asked for: a repair that cannot show its work is a repair
|
||
* nobody can trust.
|
||
*/
|
||
export interface RepairFamilyReport {
|
||
family: string
|
||
/** The family was actually examined (false = skipped; see `skipped`/`reason`). */
|
||
checked: boolean
|
||
/** Items re-posted / corrected in place — the incremental heal count. */
|
||
healed: number
|
||
/**
|
||
* What the check found missing or divergent, when it can name it: an exact
|
||
* count plus a capped sample of ids (never the whole list — a report is a
|
||
* verdict, not a dump). Absent when the family has nothing to name.
|
||
*/
|
||
missing?: { count: number; sample: string[] }
|
||
/** A full generational rebuild of this family ran (as opposed to an incremental heal). */
|
||
rebuilt?: boolean
|
||
detail?: string
|
||
/** Why the family was not checked (`checked: false`). */
|
||
skipped?: string
|
||
/** Why the outcome is what it is when neither `detail` nor `skipped` says it. */
|
||
reason?: string
|
||
/**
|
||
* The phase's own wall, in milliseconds. A repair on a production store ran
|
||
* for over thirty minutes without a single line of output; an operator had
|
||
* to read `top` to know it was alive. A receipt that cannot say WHERE the
|
||
* time went is not a receipt — every row carries its own.
|
||
*/
|
||
durationMs?: number
|
||
}
|
||
|
||
/** The full receipt returned by repairIndex(). */
|
||
export interface RepairReport {
|
||
families: RepairFamilyReport[]
|
||
healedTotal: number
|
||
durationMs: number
|
||
}
|
||
|
||
export interface BatchResult<T = any> {
|
||
successful: T[] // Successfully processed items
|
||
failed: Array<{ // Failed items with errors
|
||
item: any
|
||
error: string
|
||
}>
|
||
total: number // Total attempted
|
||
duration: number // Time taken in ms
|
||
}
|
||
|
||
// ============= Import Progress =============
|
||
|
||
/**
|
||
* Import stage enumeration
|
||
*/
|
||
export type ImportStage =
|
||
| 'detecting' // Detecting file format
|
||
| 'reading' // Reading file from disk/network
|
||
| 'parsing' // Parsing file structure (CSV rows, PDF pages, Excel sheets)
|
||
| 'extracting' // Extracting entities using AI
|
||
| 'indexing' // Creating graph nodes and relationships
|
||
| 'completing' // Final cleanup and stats
|
||
|
||
/**
|
||
* Overall import status
|
||
*/
|
||
export type ImportStatus =
|
||
| 'starting' // Initializing import
|
||
| 'processing' // Actively importing
|
||
| 'completing' // Finalizing
|
||
| 'done' // Complete
|
||
|
||
/**
|
||
* Comprehensive import progress information
|
||
*
|
||
* Provides multi-dimensional progress tracking:
|
||
* - Bytes processed (always deterministic)
|
||
* - Entities extracted and indexed
|
||
* - Stage-specific progress
|
||
* - Time estimates
|
||
* - Performance metrics
|
||
*
|
||
*/
|
||
export interface ImportProgress {
|
||
// Overall Progress
|
||
overall_progress: number // 0-100 weighted estimate across all stages
|
||
overall_status: ImportStatus // High-level status
|
||
|
||
// Current Stage
|
||
stage: ImportStage // What's happening now
|
||
stage_progress: number // 0-100 within current stage (0 if unknown)
|
||
stage_message: string // Human-readable: "Extracting entities from PDF..."
|
||
|
||
// Bytes (Always Available - most deterministic metric)
|
||
bytes_processed: number // Bytes read/processed so far
|
||
total_bytes: number // Total file size (0 if streaming/unknown)
|
||
bytes_percentage: number // Convenience: bytes_processed / total_bytes * 100
|
||
bytes_per_second?: number // Processing rate (relevant during parsing)
|
||
|
||
// Entities (Available when extraction starts)
|
||
entities_extracted: number // Entities found during AI extraction
|
||
entities_indexed: number // Entities added to Brainy graph
|
||
entities_per_second?: number // Entities/sec (relevant during extraction/indexing)
|
||
estimated_total_entities?: number // Estimated final count
|
||
estimation_confidence?: number // 0-1 confidence in estimation
|
||
|
||
// Timing
|
||
elapsed_ms: number // Time since import started
|
||
estimated_remaining_ms?: number // Estimated time remaining
|
||
estimated_total_ms?: number // Estimated total time
|
||
|
||
// Context (helps users understand what's happening)
|
||
current_item?: string // "Processing page 5 of 23"
|
||
current_file?: string // "Sheet: Q2 Sales Data"
|
||
file_number?: number // 3 (when importing multiple files)
|
||
total_files?: number // 10
|
||
|
||
// Performance Metrics (for debugging/optimization)
|
||
metrics?: {
|
||
parsing_rate_mbps?: number // MB/s during parsing
|
||
extraction_rate_entities_per_sec?: number // Entities/s during extraction
|
||
indexing_rate_entities_per_sec?: number // Entities/s during indexing
|
||
memory_usage_mb?: number // Current memory usage
|
||
peak_memory_mb?: number // Peak memory usage
|
||
}
|
||
|
||
// Backwards Compatibility (for legacy code)
|
||
current: number // Alias for entities_indexed
|
||
total: number // Alias for estimated_total_entities or 0
|
||
}
|
||
|
||
/**
|
||
* Import progress callback - backwards compatible
|
||
*
|
||
* Supports both legacy (current, total) and new (ImportProgress object) signatures
|
||
*/
|
||
export type ImportProgressCallback =
|
||
| ((progress: ImportProgress) => void)
|
||
| ((current: number, total: number) => void)
|
||
|
||
/**
|
||
* Stage weight configuration for overall progress calculation
|
||
*
|
||
* These weights reflect the typical time distribution across stages.
|
||
* Extraction is typically the slowest stage (60% of time).
|
||
*/
|
||
export interface StageWeights {
|
||
detecting: number // Default: 0.01 (1%)
|
||
reading: number // Default: 0.05 (5%)
|
||
parsing: number // Default: 0.10 (10%)
|
||
extracting: number // Default: 0.60 (60% - slowest!)
|
||
indexing: number // Default: 0.20 (20%)
|
||
completing: number // Default: 0.04 (4%)
|
||
}
|
||
|
||
/**
|
||
* Import result statistics
|
||
*/
|
||
export interface ImportStats {
|
||
graphNodesCreated: number // Entities added to graph
|
||
graphEdgesCreated: number // Relationships created
|
||
vfsFilesCreated: number // VFS files created
|
||
duration: number // Total time in ms
|
||
bytesProcessed: number // Total bytes read
|
||
averageRate: number // Average entities/sec
|
||
peakMemoryMB?: number // Peak memory usage
|
||
}
|
||
|
||
/**
|
||
* Import operation result
|
||
*/
|
||
export interface ImportResult {
|
||
success: boolean
|
||
stats: ImportStats
|
||
errors?: Array<{
|
||
stage: ImportStage
|
||
message: string
|
||
error?: any
|
||
}>
|
||
}
|
||
|
||
// ============= Advanced Operations =============
|
||
|
||
/**
|
||
* Options for brain.get() entity retrieval
|
||
*
|
||
* **Performance Optimization**:
|
||
* By default, brain.get() loads ONLY metadata (not vectors), resulting in:
|
||
* - **76-81% faster** reads (10ms vs 43ms for metadata-only)
|
||
* - **95% less bandwidth** (300 bytes vs 6KB per entity)
|
||
* - **87% less memory** (optimal for VFS and large-scale operations)
|
||
*
|
||
* **When to use includeVectors**:
|
||
* - Computing similarity on a specific entity (not search): `brain.similar({ to: entity.vector })`
|
||
* - Manual vector operations: `cosineSimilarity(entity.vector, otherVector)`
|
||
* - Inspecting embeddings for debugging
|
||
*
|
||
* **When NOT to use includeVectors** (metadata-only is sufficient):
|
||
* - VFS operations (readFile, stat, readdir) - 100% of cases
|
||
* - Existence checks: `if (await brain.get(id))`
|
||
* - Metadata inspection: `entity.metadata`, `entity.data`, `entity.type`
|
||
* - Relationship traversal: `brain.related({ from: id })`
|
||
* - Search operations: `brain.find()` generates embeddings automatically
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* // ✅ FAST (default): Metadata-only - 10ms, 300 bytes
|
||
* const entity = await brain.get(id)
|
||
* console.log(entity.data, entity.metadata) // ✅ Available
|
||
* console.log(entity.vector) // Empty Float32Array (stub)
|
||
*
|
||
* // ✅ FULL: Load vectors when needed - 43ms, 6KB
|
||
* const fullEntity = await brain.get(id, { includeVectors: true })
|
||
* const similarity = cosineSimilarity(fullEntity.vector, otherVector)
|
||
*
|
||
* // ✅ VFS automatically uses fast path (no change needed)
|
||
* await vfs.readFile('/file.txt') // 53ms → 10ms (81% faster)
|
||
* ```
|
||
*
|
||
*/
|
||
export interface GetOptions {
|
||
/**
|
||
* Include 384-dimensional vector embeddings in the response
|
||
*
|
||
* **Default: false** (metadata-only for 76-81% speedup)
|
||
*
|
||
* Set to `true` when you need to:
|
||
* - Compute similarity on this specific entity's vector
|
||
* - Perform manual vector operations
|
||
* - Inspect embeddings for debugging
|
||
*
|
||
* **Note**: Search operations (`brain.find()`) generate vectors automatically,
|
||
* so you don't need this flag for search. Only for direct vector operations
|
||
* on a retrieved entity.
|
||
*
|
||
* @default false
|
||
*/
|
||
includeVectors?: boolean
|
||
}
|
||
|
||
/**
|
||
* Graph traversal parameters
|
||
*/
|
||
export interface TraverseParams {
|
||
from: string | string[] // Starting node(s)
|
||
direction?: 'out' | 'in' | 'both' // Traversal direction
|
||
types?: VerbType[] // Edge types to follow
|
||
depth?: number // Max depth (default: 2)
|
||
strategy?: 'bfs' | 'dfs' // Breadth or depth first
|
||
filter?: (entity: Entity, depth: number, path: string[]) => boolean
|
||
limit?: number // Max nodes to visit
|
||
}
|
||
|
||
// ============= Aggregation Engine Types =============
|
||
|
||
/**
|
||
* Supported aggregation operations
|
||
*/
|
||
export type AggregationOp =
|
||
| 'sum'
|
||
| 'count'
|
||
| 'avg'
|
||
| 'min'
|
||
| 'max'
|
||
| 'stddev'
|
||
| 'variance'
|
||
/** Exact percentile (requires `p` in `[0,1]` on the metric def) — value-multiset, delete-safe. */
|
||
| 'percentile'
|
||
/** Exact count of distinct values — value-multiset, delete-safe. */
|
||
| 'distinctCount'
|
||
|
||
/**
|
||
* Time window granularity for GROUP BY time dimensions
|
||
*/
|
||
export type TimeWindowGranularity =
|
||
| 'hour' | 'day' | 'week' | 'month' | 'quarter' | 'year'
|
||
| { seconds: number }
|
||
|
||
/**
|
||
* A GROUP BY dimension — one of:
|
||
* - a plain metadata field name (string),
|
||
* - a time-windowed field (`{ field, window }`), or
|
||
* - an **unnest** field (`{ field, unnest: true }`): the field holds an array, and the entity
|
||
* contributes once to a group per distinct element (e.g. `tags: string[]` → tag frequency).
|
||
* An entity whose unnest field is missing or an empty array contributes to no group.
|
||
*/
|
||
export type GroupByDimension =
|
||
| string
|
||
| { field: string; window: TimeWindowGranularity }
|
||
| { field: string; unnest: true }
|
||
|
||
/**
|
||
* Source filter for which entities feed into an aggregate
|
||
*/
|
||
export interface AggregateSource {
|
||
/** Filter by entity type(s) */
|
||
type?: NounType | NounType[]
|
||
/**
|
||
* Metadata filter — same syntax and field-addressing law as find()'s
|
||
* `where` (see the `@remarks` on {@link FindParams}).
|
||
*/
|
||
where?: Record<string, unknown>
|
||
/** Multi-tenancy service filter */
|
||
service?: string
|
||
}
|
||
|
||
/**
|
||
* Full aggregate definition — registered via brain.defineAggregate()
|
||
*/
|
||
export interface AggregateDefinition {
|
||
/** Unique name for this aggregate (used in queries) */
|
||
name: string
|
||
/** Which entities contribute to this aggregate */
|
||
source: AggregateSource
|
||
/**
|
||
* Dimensions to group by — field names follow the same field-addressing
|
||
* law as find()'s `where` / `orderBy` (see the `@remarks` on
|
||
* {@link FindParams}).
|
||
*/
|
||
groupBy: GroupByDimension[]
|
||
/** Named metrics to compute */
|
||
metrics: Record<string, AggregateMetricDef>
|
||
/** Control materialization of results as NounType.Measurement entities */
|
||
materialize?: boolean | { debounceMs?: number; trackSources?: boolean }
|
||
}
|
||
|
||
/**
|
||
* Single metric definition within an aggregate
|
||
*/
|
||
export interface AggregateMetricDef {
|
||
/** Aggregation operation */
|
||
op: AggregationOp
|
||
/** Metadata field to aggregate (required for all ops except count) */
|
||
field?: string
|
||
/** Percentile fraction in `[0, 1]` — required when `op === 'percentile'`, ignored otherwise. */
|
||
p?: number
|
||
}
|
||
|
||
/**
|
||
* Internal running state for a single metric.
|
||
* Tracks enough to compute all operations incrementally.
|
||
*/
|
||
export interface MetricState {
|
||
sum: number
|
||
count: number
|
||
min: number
|
||
max: number
|
||
/** Running M2 for Welford's online variance (sum of squared differences from mean) */
|
||
m2?: number
|
||
/**
|
||
* Value multiset (String(value) → occurrence count) for exact percentile + distinctCount.
|
||
* Maintained only for `percentile`/`distinctCount` metrics. JSON-serializable; mirrors
|
||
* Cor's `value_counts` so JS and native agree bit-for-bit.
|
||
*/
|
||
valueCounts?: Record<string, number>
|
||
}
|
||
|
||
/**
|
||
* Internal state for one aggregate group (one combination of group key values)
|
||
*/
|
||
export interface AggregateGroupState {
|
||
/** The group key values (e.g., { category: 'food', period: '2024-01' }) */
|
||
groupKey: Record<string, string | number>
|
||
/** Running metric states keyed by metric name */
|
||
metrics: Record<string, MetricState>
|
||
/** Entity ID of the materialized Measurement entity (if materialized) */
|
||
materializedEntityId?: string
|
||
/** Timestamp of last update */
|
||
lastUpdated: number
|
||
}
|
||
|
||
/**
|
||
* Query parameters for reading aggregate results
|
||
*/
|
||
export interface AggregateQueryParams {
|
||
/** Name of the aggregate to query */
|
||
name: string
|
||
/**
|
||
* Filter aggregate groups by their key values — same field-addressing
|
||
* law as find() (see the `@remarks` on {@link FindParams}).
|
||
*/
|
||
where?: Record<string, unknown>
|
||
/**
|
||
* Filter groups by their computed METRIC values (SQL HAVING). Same BFO operators as
|
||
* `where`, but applied to the derived metric results plus `count`, e.g.
|
||
* `{ revenue: { greaterThan: 1000 } }`. Evaluated per group (O(groups), independent of
|
||
* entity count), before sort/pagination. Metric names and `count` are looked up
|
||
* directly, not field-addressed; a group-KEY field used here follows the same
|
||
* field-addressing law as find() (see the `@remarks` on {@link FindParams}).
|
||
*/
|
||
having?: Record<string, unknown>
|
||
/**
|
||
* Sort by metric name (a key from `metrics`, looked up directly) or by a
|
||
* group key field — a group key field follows the same field-addressing
|
||
* law as find()'s `orderBy` (see the `@remarks` on {@link FindParams}).
|
||
*/
|
||
orderBy?: string
|
||
/** Sort direction */
|
||
order?: 'asc' | 'desc'
|
||
/** Max results */
|
||
limit?: number
|
||
/** Skip N results */
|
||
offset?: number
|
||
}
|
||
|
||
/**
|
||
* A single aggregate result row
|
||
*/
|
||
export interface AggregateResult {
|
||
/** Group key values for this row */
|
||
groupKey: Record<string, string | number>
|
||
/** Computed metric values (derived from MetricState based on op) */
|
||
metrics: Record<string, number>
|
||
/** Total entity count in this group */
|
||
count: number
|
||
/** Entity ID of materialized Measurement (if materialized) */
|
||
entityId?: string
|
||
}
|
||
|
||
/**
|
||
* Provider interface for Cor-accelerated aggregation.
|
||
* When registered as 'aggregation' provider, Brainy delegates to this.
|
||
*/
|
||
export interface AggregationProvider {
|
||
/** Register an aggregate definition (caches compiled definition for hot path) */
|
||
defineAggregate?(def: AggregateDefinition): void
|
||
|
||
/** Remove a registered aggregate definition */
|
||
removeAggregate?(name: string): void
|
||
|
||
/** Incrementally update aggregation state when an entity changes */
|
||
incrementalUpdate(
|
||
name: string,
|
||
def: AggregateDefinition,
|
||
entity: Record<string, unknown>,
|
||
op: 'add' | 'update' | 'delete',
|
||
prev?: Record<string, unknown>
|
||
): AggregateGroupState[]
|
||
|
||
/** Compute the group key for a given entity */
|
||
computeGroupKey(
|
||
entity: Record<string, unknown>,
|
||
groupBy: GroupByDimension[]
|
||
): Record<string, string | number>
|
||
|
||
/** Rebuild an entire aggregate from scratch */
|
||
rebuildAggregate(
|
||
def: AggregateDefinition,
|
||
entities: Array<Record<string, unknown>>
|
||
): Map<string, AggregateGroupState>
|
||
|
||
/** Query aggregate state with filtering/sorting/pagination */
|
||
queryAggregate(
|
||
state: Map<string, AggregateGroupState>,
|
||
params: AggregateQueryParams
|
||
): AggregateResult[]
|
||
|
||
/** Restore previously serialized state (called during init) */
|
||
restoreState?(data: string): void
|
||
|
||
/** Serialize internal state for persistence (called during flush) */
|
||
serializeState?(): string
|
||
|
||
/**
|
||
* Bake the committed generation into the provider's own state envelope
|
||
* before {@link serializeState} (called during flush, immediately prior).
|
||
* Lets a native-side reopen verify the envelope's honesty independently of
|
||
* the host's wrapper stamp. Optional — providers without it rely on the
|
||
* host wrapper's `sourceGeneration` alone.
|
||
*/
|
||
noteSourceGeneration?(generation: number): void
|
||
}
|
||
|
||
// ============= Configuration =============
|
||
|
||
/**
|
||
* Integration Hub configuration
|
||
*
|
||
* Enables external tool integrations: Excel, Power BI, Google Sheets, etc.
|
||
* Works in all environments with zero external dependencies.
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* // Enable all integrations with defaults
|
||
* new Brainy({ integrations: true })
|
||
*
|
||
* // Custom configuration
|
||
* new Brainy({
|
||
* integrations: {
|
||
* basePath: '/api/v1',
|
||
* enable: ['odata', 'sheets']
|
||
* }
|
||
* })
|
||
* ```
|
||
*/
|
||
export interface IntegrationsConfig {
|
||
/** Base path for all integration endpoints (default: '') */
|
||
basePath?: string
|
||
|
||
/** Which integrations to enable (default: all) */
|
||
enable?: ('odata' | 'sheets' | 'sse' | 'webhooks')[] | 'all'
|
||
|
||
/** Per-integration config overrides */
|
||
config?: {
|
||
odata?: { basePath?: string }
|
||
sheets?: { basePath?: string }
|
||
sse?: { basePath?: string; heartbeatInterval?: number }
|
||
webhooks?: { maxRetries?: number }
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Operator-facing summary returned by `brain.stats()`. Designed to fit in a
|
||
* terminal screen so an incident responder can read it at a glance: counts,
|
||
* mode, lock owner, indexed fields, and index health flags.
|
||
*/
|
||
export interface BrainyStats {
|
||
/** Whether this instance can mutate. `reader` is set by `openReadOnly()` and `asOf()`. */
|
||
mode: 'writer' | 'reader'
|
||
/** Total entity (noun) count across all types. */
|
||
entityCount: number
|
||
/** Breakdown by NounType name (`person`, `event`, ...). Zero-count types omitted. */
|
||
entitiesByType: Record<string, number>
|
||
/** Total relationship (verb) count across all types. */
|
||
relationCount: number
|
||
/** Breakdown by VerbType name. Zero-count types omitted. */
|
||
relationsByType: Record<string, number>
|
||
/** Indexed metadata field names known to the metadata index. */
|
||
fieldRegistry: string[]
|
||
/** Per-index health flags. `true` = index has entries OR no entities exist yet. */
|
||
indexHealth: {
|
||
/** Vector index health (8.0 — open-core JS HNSW path or a native acceleration provider). */
|
||
vector: boolean
|
||
metadata: boolean
|
||
graph: boolean
|
||
}
|
||
/** Storage backend info — `backend` is the adapter class name. */
|
||
storage: {
|
||
backend: string
|
||
rootDir?: string
|
||
}
|
||
/**
|
||
* Writer lock metadata if one is currently held on this directory. Present
|
||
* for both writer instances (their own lock) and readers inspecting a
|
||
* directory with a live writer.
|
||
*/
|
||
writerLock?: {
|
||
pid: number
|
||
hostname: string
|
||
startedAt: string
|
||
lastHeartbeat: string
|
||
version: string
|
||
rootDir?: string
|
||
}
|
||
/** The Brainy library version this instance was built against. */
|
||
version: string
|
||
}
|
||
|
||
/**
|
||
* Brainy configuration
|
||
*/
|
||
/**
|
||
* Structured progress of the one-time, automatic 7.x → 8.0 migration (the
|
||
* coordinated LOCK). Relayed verbatim from the native provider's optional
|
||
* `migrationStatus()` and surfaced on `getIndexStatus().migration` while the
|
||
* brain is upgrading. Every field is optional — a provider may report only a
|
||
* phase, or nothing (in which case `getIndexStatus()` shows `migrating: true`
|
||
* plus `elapsedMs` alone).
|
||
*/
|
||
export interface MigrationProgress {
|
||
/** Coarse stage, e.g. `'rebuilding'` | `'verifying'`. */
|
||
phase?: string
|
||
/** Which derived index is currently rebuilding. */
|
||
index?: 'metadata' | 'vector' | 'graph'
|
||
/** Overall progress, 0–100. */
|
||
percent?: number
|
||
/** Canonical entities processed so far. */
|
||
entitiesDone?: number
|
||
/** Total canonical entities to process. */
|
||
entitiesTotal?: number
|
||
/** When the migration was first observed (epoch ms). */
|
||
startedAt?: number
|
||
/** Milliseconds elapsed since the migration was first observed. */
|
||
elapsedMs?: number
|
||
}
|
||
|
||
export interface BrainyConfig {
|
||
/**
|
||
* Storage configuration: either a config object resolved through the
|
||
* storage factory, or a pre-constructed adapter instance (used directly —
|
||
* e.g. `storage: new MemoryStorage()`; Brainy's historical-query
|
||
* materializer hands a pre-populated instance through this path).
|
||
*/
|
||
storage?:
|
||
| {
|
||
/**
|
||
* Storage backend. Optional — a top-level `path` implies `'filesystem'`,
|
||
* so `storage: { path: '/data' }` works without it. `'auto'` (the
|
||
* default) picks filesystem on Node, memory in a browser.
|
||
*/
|
||
type?: 'auto' | 'memory' | 'filesystem'
|
||
/**
|
||
* **Canonical** directory for filesystem storage. The rest of the API
|
||
* already speaks `path` (`persist(path)`, `Brainy.load(path)`,
|
||
* `asOf(path)`, `restore(path)`). Specifying it implies
|
||
* `type: 'filesystem'`. Passed through to plugin-provided storage
|
||
* factories (e.g. native mmap providers) so they resolve the same root.
|
||
* @example
|
||
* new Brainy({ storage: { path: '/var/lib/app/data' } })
|
||
*/
|
||
path?: string
|
||
/**
|
||
* @deprecated REMOVED in 8.0 — passing `rootDirectory` THROWS. Use the
|
||
* canonical {@link path}.
|
||
*/
|
||
rootDirectory?: string
|
||
/**
|
||
* @deprecated REMOVED in 8.0 — a nested `options.path` / `options.rootDirectory`
|
||
* THROWS. Use the top-level {@link path}.
|
||
*/
|
||
options?: any
|
||
/**
|
||
* @deprecated REMOVED in 8.0 — `fileSystemStorage.path` / `.rootDirectory`
|
||
* THROWS. Use the top-level {@link path}.
|
||
*/
|
||
fileSystemStorage?: { path?: string; rootDirectory?: string; [key: string]: any }
|
||
}
|
||
| StorageAdapter
|
||
|
||
/**
|
||
* RE-MEANT (the health-gate contract): `init()` (open) always verifies the
|
||
* durable generation of every derived index, and a needed rebuild ALWAYS
|
||
* runs at open — it is never deferred to the first read, regardless of
|
||
* dataset size or this flag. There is no first-query lazy-build path
|
||
* anymore: a read that finds a provider not serving throws a typed
|
||
* `*NotReadyError` rather than building anything (see
|
||
* `assessProviderHealth` / the read gate in `brainy.ts`). Setting this
|
||
* `true` no longer defers index construction to the first query — it has
|
||
* no effect on WHEN a needed rebuild runs. Full manual control over
|
||
* rebuilds remains available via `repairIndex({ rebuild: [...] })`.
|
||
*/
|
||
disableAutoRebuild?: boolean
|
||
|
||
/**
|
||
* The floor (ms) of the apply-phase transaction budget's scaling formula:
|
||
* `max(transactionBudgetFloorMs, opCount × 2 000)`. The budget governs
|
||
* whether a `transact()` / single-op write's NEXT internal operation may
|
||
* **start** — never whether already-completed work gets rolled back after
|
||
* the fact (a single-op write can never time out post-hoc: it either runs
|
||
* or it commits). A trip mid-batch still rolls back every applied operation
|
||
* atomically and throws a `TransactionTimeoutError` that is
|
||
* retryable-with-latch, never hot-retry (see its `retryable` and
|
||
* `hotRetryUnsafe` fields); only the floor of the formula is configurable
|
||
* here.
|
||
*
|
||
* Raise this when a cold store's first writes after a restart legitimately
|
||
* take longer than 30s per operation (e.g. page-cache-cold canonical writes
|
||
* on a large brain) so a bulk `transact()` batch gets a proportionally
|
||
* larger runway instead of tripping mid-batch. Lower it to fail faster on a
|
||
* latency-sensitive write path. Default: `30000` (30 s) — unchanged from
|
||
* pre-8.9 behavior.
|
||
*/
|
||
transactionBudgetFloorMs?: number
|
||
|
||
/**
|
||
* How long (ms) an operation **waits** on the coordinated 7.x → 8.0 migration
|
||
* before throwing a retryable `MigrationInProgressError`.
|
||
*
|
||
* IMPORTANT: this bounds the *caller's wait*, NOT the migration. The migration
|
||
* itself is **unbounded** — a native provider rebuilds a billion-scale brain
|
||
* for as long as it needs, and this timeout never interrupts it. While the
|
||
* rebuild runs, reads and writes (and `init()` before it serves) block so no
|
||
* operation touches a half-built index:
|
||
* - **Small/medium upgrade (< this window):** the caller waits, then resumes
|
||
* transparently — no error.
|
||
* - **Large upgrade (> this window):** the caller gets a retryable
|
||
* `MigrationInProgressError` (the rebuild continues in the background);
|
||
* `getIndexStatus().migrating` — never gated — is the readiness-probe signal
|
||
* that maps to HTTP 503 + Retry-After so an orchestrator waits rather than
|
||
* routing traffic in.
|
||
*
|
||
* Raise it for a latency-tolerant batch job (wait longer / effectively wait
|
||
* through); lower it to fail fast on a request path. Default: `30000` (30 s).
|
||
*/
|
||
migrationWaitTimeoutMs?: number
|
||
|
||
/**
|
||
* Take an automatic **pre-upgrade backup** before a one-time 7.x → 8.0
|
||
* migration rebuilds the derived indexes, and remove it once the upgrade
|
||
* verifies (retain it on failure, for rollback). On the filesystem adapter
|
||
* this is a **hard-link snapshot** of the brain directory — near-zero cost and
|
||
* space (shared inodes; the store is immutable-by-rename), even at scale.
|
||
* The migration is already structurally safe (it only reads canonical records,
|
||
* derived indexes are fully reconstructable, and a failed upgrade self-heals on
|
||
* re-open), so this is catastrophe-insurance against a migration *bug*, not a
|
||
* data-loss guard — and NOT a substitute for an off-device backup. No-op for
|
||
* non-filesystem storage and for a brain with no persisted data.
|
||
* Default: `true`. Set `false` to opt out (e.g. you run your own backup).
|
||
*/
|
||
migrationBackup?: boolean
|
||
|
||
/**
|
||
* Vector index configuration (Brainy 8.0).
|
||
*
|
||
* Two knobs. No escape hatch. The algorithm-internal HNSW knobs
|
||
* (`M`, `efConstruction`, `efSearch`, `ml`, …) and DiskANN knobs
|
||
* (`pqM`, `searchListSize`, `alpha`, …) are deliberately not exposed —
|
||
* the `recall` preset covers the legitimate quality/latency tradeoff
|
||
* range, and the open-core defaults match Brainy 7.x's defaults
|
||
* byte-for-byte so a `'balanced'`-default upgrade is a no-op.
|
||
*
|
||
* **Closed-form contract** locked in handoff thread
|
||
* BRAINY-8.0-RENAME-COORDINATION § A.2 + § G.2 (cor-confirmed 2026-06-09).
|
||
*/
|
||
vector?: {
|
||
/**
|
||
* Recall preset.
|
||
* - `'fast'` — minimum-latency search, accepts lower recall
|
||
* - `'balanced'` (default) — Brainy 7.x defaults, the right pick for almost everyone
|
||
* - `'accurate'` — maximum recall, accepts higher latency
|
||
*
|
||
* Means the same thing whether the open-core JS HNSW path is in play
|
||
* or a native vector provider has taken over the `'vector'` provider
|
||
* key — Brainy translates to HNSW knobs; native providers translate
|
||
* to their own (e.g. DiskANN's `defaultLSearch` / `defaultPaddingFactor`).
|
||
*/
|
||
recall?: 'fast' | 'balanced' | 'accurate'
|
||
|
||
/**
|
||
* Vector persistence mode. `'immediate'` writes per-noun graph state on
|
||
* every `add()`; durable but slower. `'deferred'` writes only on
|
||
* `flush()` / `close()`; faster bulk ingest.
|
||
*
|
||
* **Auto-selected from the storage adapter when omitted:** `'immediate'`
|
||
* on filesystem storage (durability is the point of a persistent
|
||
* backend), `'deferred'` on memory storage (nothing survives the process
|
||
* anyway, so per-add persistence writes are pure overhead).
|
||
*
|
||
* Brainy 7.x called this `hnswPersistMode` at the top level. The 8.0
|
||
* surface folds it under `config.vector` alongside `recall`.
|
||
*/
|
||
persistMode?: 'immediate' | 'deferred'
|
||
}
|
||
|
||
/**
|
||
* Generational-history retention policy (8.0 MVCC — the `retention` knob).
|
||
*
|
||
* Under Model-B EVERY write (`transact()` AND single-op `add`/`update`/
|
||
* `remove`/`relate`) produces an immutable generation record-set serving
|
||
* historical reads (`asOf()`, pinned `Db` values). Without compaction those
|
||
* accumulate, so Brainy **auto-compacts at `close()`** (time-bounded per
|
||
* pass; 8.9.0 removed compaction from `flush()` — flush is durability work
|
||
* and never pays maintenance costs). A long-lived writer that never closes
|
||
* accumulates history until its next explicit `compactHistory()` call.
|
||
* Live `Db` pins are ALWAYS exempt from reclamation, in every mode.
|
||
*
|
||
* Modes:
|
||
* - **unset → ADAPTIVE (default):** the retention horizon tracks
|
||
* disk/RAM pressure. A machine-level byte budget governs how much history
|
||
* is kept — driven by `budgetBytes` when a coordinator (e.g. cor's
|
||
* `ResourceManager`, fair-shared across co-located instances) sets it, else
|
||
* a local `os.freemem`/filesystem-free probe. Oldest-unpinned history is
|
||
* reclaimed when the budget is exceeded. Zero-config.
|
||
* - **`'all'` → unbounded:** never reclaim history (index compaction for
|
||
* query speed still runs — the decouple). The legacy keep-everything
|
||
* behavior, now explicit opt-in.
|
||
* - **`{ maxGenerations?, maxAge?, maxBytes? }` → explicit CAPS:** reclaim
|
||
* the oldest unpinned generations while ANY supplied cap is exceeded
|
||
* (predictable ops). `maxAge` in ms; `maxBytes` total history bytes.
|
||
*
|
||
* `autoCompact: false` disables the automatic close() compaction (manage
|
||
* manually via `brain.compactHistory()`). `budgetBytes` is the settable
|
||
* adaptive byte budget a coordinator drives (also via `brain.setRetentionBudget()`).
|
||
* Long-term archives belong in `db.persist(path)` snapshots, which compaction
|
||
* never touches.
|
||
*/
|
||
retention?:
|
||
| 'all'
|
||
| 'adaptive'
|
||
| {
|
||
/** Keep at most this many recent committed generations (cap). */
|
||
maxGenerations?: number
|
||
/** Keep generations committed within this window in ms (cap). */
|
||
maxAge?: number
|
||
/** Keep total generational-history bytes at or below this (cap). */
|
||
maxBytes?: number
|
||
/** Adaptive byte budget for this brain, driven by a coordinator (e.g. cor). */
|
||
budgetBytes?: number
|
||
/** Run compaction automatically at close() (default: true; 8.9.0 — flush() never compacts). */
|
||
autoCompact?: boolean
|
||
}
|
||
|
||
// Memory management options
|
||
maxQueryLimit?: number // Override auto-detected query result limit (max: 100000)
|
||
reservedQueryMemory?: number // Memory reserved for queries in bytes (e.g., 1073741824 = 1GB)
|
||
|
||
/**
|
||
* Controls whether `init()` starts a BACKGROUND warm of the WASM embedding
|
||
* engine.
|
||
*
|
||
* **Adaptive default (8.0, background since the open-path fix):** when
|
||
* omitted, `init()` STARTS a background warm of the engine whenever the
|
||
* WASM embedder is the *active* one — i.e. no native `'embeddings'`
|
||
* provider is registered — and this instance is a writer (not
|
||
* `mode: 'reader'`) running outside unit tests. The WASM module (≈93MB with
|
||
* the embedded model) takes 90-140s to compile on throttled CPUs — but
|
||
* `init()` never awaits that compile. It only starts it, so N concurrent
|
||
* opens no longer serialize on the one process-global engine singleton.
|
||
* The first `embed()` call then waits for whichever finishes first: the
|
||
* background warm (if still running) or its own fresh init (if the warm
|
||
* never started, e.g. `eagerEmbeddings: false`) — both paths converge on
|
||
* the SAME shared promise inside the engine singleton, so the vector is
|
||
* always correct; only the timing of who pays the wait differs.
|
||
*
|
||
* The adaptive path skips itself automatically when a native embeddings
|
||
* provider owns embeddings, in reader-mode (readers query existing vectors
|
||
* and never embed), and in unit-test mode (kept fast via the mock embedder).
|
||
*
|
||
* - `true` — force the background warm to start during `init()` (the
|
||
* adaptive default already does this for the active-embedder writer
|
||
* case; set it explicitly to be unambiguous).
|
||
* - `false` — no warm at all. Fully lazy: the first `embed()` call pays the
|
||
* full cold-compile cost inline, on whichever request triggers it.
|
||
*/
|
||
eagerEmbeddings?: boolean
|
||
|
||
/**
|
||
* When `true`, `init()` **awaits `warm()`** (see {@link Brainy.warm}) before
|
||
* it resolves — blocking startup until the vector index, metadata index,
|
||
* and graph adjacency have all faulted their backing storage in. This is a
|
||
* deliberate operator opt-in trade: startup takes longer, but the FIRST
|
||
* request after a cold restart runs at steady-state cost instead of paying
|
||
* page-cache-miss / demand-load latency on the critical path.
|
||
*
|
||
* Runs AFTER `init()`'s own sequence completes (index construction, crash
|
||
* recovery, migrations, VFS bootstrap) — never in place of any of it — so
|
||
* `warm()` always operates on a fully-initialized brain.
|
||
*
|
||
* Default: `false` (lazy — the pre-8.9 behavior: indexes demand-load on
|
||
* first access).
|
||
*/
|
||
warmOnOpen?: boolean
|
||
|
||
// Plugin configuration
|
||
// Controls which plugins are loaded during init().
|
||
// - undefined (default): guarded auto-detection of the first-party
|
||
// accelerator (@soulcraft/cor) — installing the package IS the opt-in.
|
||
// Not installed → plain brainy, silently. Installed → it loads and
|
||
// announces itself. Installed but broken → init() THROWS (an installed
|
||
// accelerator never silently vanishes behind the JS engines).
|
||
// - false / []: no plugins, no detection (explicit opt-out)
|
||
// - ['@soulcraft/cor']: load exactly these packages; a listed plugin that
|
||
// fails to load or is invalid THROWS (loud, never a silent JS fallback)
|
||
plugins?: string[] | false
|
||
|
||
// Logging configuration
|
||
verbose?: boolean // Enable verbose logging
|
||
silent?: boolean // Suppress all logging output
|
||
|
||
// Integration Hub
|
||
// Enable external tool integrations: Excel, Power BI, Google Sheets, etc.
|
||
// - true: Enable all integrations with default paths
|
||
// - false/undefined: Disable integrations (default)
|
||
// - IntegrationsConfig: Custom configuration
|
||
integrations?: boolean | IntegrationsConfig
|
||
|
||
// Migration configuration
|
||
// - true/undefined (default): Automatically run pending data migrations during
|
||
// init() for small datasets (<10K entities). Larger datasets defer with a
|
||
// notice — call brain.migrate() explicitly (optionally with backupTo).
|
||
// - false: Never auto-run; log a notice when pending migrations exist.
|
||
autoMigrate?: boolean
|
||
|
||
/**
|
||
* Brain-wide subtype enforcement mode.
|
||
*
|
||
* **Default-on since 8.0.0** (`undefined` resolves to `true`; in 7.30.x this
|
||
* was an opt-in flag defaulting to `false`).
|
||
*
|
||
* - `true` / `undefined` (default): every `add()` / `addMany()` / `update()` /
|
||
* `relate()` / `relateMany()` / `updateRelation()` rejects writes where the
|
||
* entity's NounType (or relationship's VerbType) has no non-empty `subtype`
|
||
* value. Brainy's own VFS infrastructure writes (`metadata.isVFSEntity` /
|
||
* `metadata.isVFS`) bypass the check.
|
||
* - `{ except: [NounType.Thing, NounType.Custom] }`: same as `true`, but the
|
||
* listed types are allowed through without a subtype. Use for genuine
|
||
* catch-all types where no subtype makes sense.
|
||
* - `false`: disable the brain-wide check entirely. Last-resort escape hatch
|
||
* for opening pre-8.0 data — run `brain.audit()` to find the gaps, back-fill
|
||
* with `brain.fillSubtypes(rules)`, then remove the opt-out so the default
|
||
* enforcement protects new writes.
|
||
*
|
||
* Per-type registrations always compose with the brain-wide flag — a type
|
||
* registered with `requireSubtype(type, { required: true })` is always
|
||
* enforced regardless of this flag.
|
||
*/
|
||
requireSubtype?: boolean | { except: Array<import('../types/graphTypes.js').NounType | import('../types/graphTypes.js').VerbType> }
|
||
|
||
/**
|
||
* Process role for multi-process safety on filesystem storage.
|
||
*
|
||
* - `'writer'` (default): acquires an exclusive lock on the storage directory at
|
||
* `init()` and refuses to open if another live writer holds the lock. Required
|
||
* for any instance that calls `add`/`update`/`remove`/`relate` or any other
|
||
* mutation. Released on `close()` and on process exit/SIGINT/SIGTERM.
|
||
* - `'reader'`: opens without acquiring the writer lock. Coexists with a live
|
||
* writer and with other readers. All mutation methods throw
|
||
* `Cannot mutate a read-only Brainy instance`. Use `Brainy.openReadOnly()`
|
||
* as a convenience factory.
|
||
*
|
||
* See `docs/concepts/multi-process.md` for the full coordination model.
|
||
*/
|
||
mode?: 'writer' | 'reader'
|
||
|
||
/**
|
||
* Bypass the writer-lock check at init even when another writer holds the lock.
|
||
* Use only when you know the existing lock is stale and stale-detection
|
||
* (PID liveness + heartbeat) cannot prove it. Logs a warning regardless.
|
||
*/
|
||
force?: boolean
|
||
|
||
/**
|
||
* THE ENGINE OWNS ITS FLUSH CADENCE (the persistence policy —
|
||
* SELF-ENGINE-LIFECYCLE-SPRINT, David-directed: "why do we need manual
|
||
* flushes at all?"). Under `'auto'` (the DEFAULT) the engine schedules
|
||
* single-flight background flushes itself — triggered by write count,
|
||
* elapsed time, and idle — so callers NEVER call `flush()` in a hot path
|
||
* (a production consumer's 829 per-write flushes convoyed into 45–66s
|
||
* write walls; the cadence belongs to the layer that can see dirty-node
|
||
* counts and IO pressure). `flush()` remains public as an awaitable
|
||
* durability BARRIER for the rare "must be on disk before I proceed"
|
||
* moment — calling it is never wrong, just no longer necessary.
|
||
*
|
||
* RECOVERY SEMANTICS (the documented promise): canonical records are
|
||
* durable per-write, independent of this policy — a crash between
|
||
* background flushes loses NO data. What a flush persists is DERIVED
|
||
* state (index postings, deferred HNSW nodes, counters, aggregation
|
||
* stamps); after a crash, derived state converges at the next open from
|
||
* canonical records (epoch machinery + incremental aggregation catch-up),
|
||
* paying a bounded catch-up cost proportional to the un-flushed window —
|
||
* never data loss.
|
||
*
|
||
* `'manual'` restores the pre-9.1 behavior: the engine never flushes on
|
||
* its own (except at `close()`); the caller owns the cadence.
|
||
*/
|
||
/**
|
||
* Storage-authority posture at open (10.0.0+ fleet default: `'adopt'`).
|
||
*
|
||
* `'adopt'` — a brain with NO stored authority artifact adopts LOG
|
||
* AUTHORITY at open, oracle-gated: the verification oracle replays the
|
||
* generation log against stored truth; curable divergences (pre-log
|
||
* rows, witness drift) are baseline-backfilled; the brain flips ONLY on
|
||
* a green verdict and writes the durable per-brain switch. On green,
|
||
* writes become durable-at-ack (group-committed log fsync covers every
|
||
* ack). A brain whose oracle cannot go green STAYS tree-authoritative,
|
||
* says so loudly, and records the refusal — never a silent half-state.
|
||
*
|
||
* `'defer'` — the explicit opt-out: no automatic adoption; the brain
|
||
* stays tree-authoritative until `adoptLogAuthority()` is called. The
|
||
* pre-10 behavior, documented for operators who stage their own flips.
|
||
*
|
||
* A STORED artifact always wins over this setting (checked-at-open law):
|
||
* an already-flipped brain stays flipped; an explicitly-recorded tree
|
||
* posture is honored until an operator re-runs adoption.
|
||
*/
|
||
logAuthority?: 'adopt' | 'defer'
|
||
|
||
persistence?: {
|
||
policy?: 'auto' | 'manual'
|
||
/** Background flush after this many committed writes (default 512). */
|
||
flushEveryWrites?: number
|
||
/** Background flush when this much time has passed since the last flush, checked at write time (default 30_000). */
|
||
flushIntervalMs?: number
|
||
/** Background flush after the store goes quiet for this long with dirty state (default 2_000). */
|
||
flushOnIdleMs?: number
|
||
}
|
||
}
|
||
|
||
// ============= Neural API Types =============
|
||
|
||
/**
|
||
* Neural similarity parameters
|
||
*/
|
||
export interface NeuralSimilarityParams {
|
||
between?: [any, any] // Compare two items
|
||
items?: any[] // Compare multiple items
|
||
explain?: boolean // Return detailed breakdown
|
||
}
|
||
|
||
/**
|
||
* Neural clustering parameters
|
||
*/
|
||
export interface NeuralClusterParams {
|
||
items?: string[] | Entity[] // Items to cluster (or all)
|
||
algorithm?: 'hierarchical' | 'kmeans' | 'dbscan' | 'spectral'
|
||
params?: {
|
||
k?: number // Number of clusters (kmeans)
|
||
threshold?: number // Distance threshold (hierarchical)
|
||
epsilon?: number // DBSCAN epsilon
|
||
minPoints?: number // DBSCAN min points
|
||
}
|
||
visualize?: boolean // Return visualization data
|
||
}
|
||
|
||
/**
|
||
* Neural anomaly detection parameters
|
||
*/
|
||
export interface NeuralAnomalyParams {
|
||
threshold?: number // Standard deviations (default: 2.5)
|
||
type?: NounType // Check specific type
|
||
method?: 'isolation' | 'lof' | 'statistical' | 'autoencoder'
|
||
returnScores?: boolean // Return anomaly scores
|
||
}
|
||
|
||
// ============= Content Extraction Types =============
|
||
|
||
/**
|
||
* Detected content type for smart text extraction
|
||
*
|
||
*/
|
||
export type ContentType = 'plaintext' | 'richtext-json' | 'html' | 'markdown'
|
||
|
||
/**
|
||
* Content category for extracted segments
|
||
*
|
||
* Universal categories that work across documents, code, and UI:
|
||
* - 'title': Names the subject — headings, identifiers, labels, JSON keys
|
||
* - 'annotation': Human explanation — comments, docstrings, captions, alt text
|
||
* - 'content': Body substance — paragraphs, list items, flowing text
|
||
* - 'value': Data literals — strings, numbers, form values, error messages
|
||
* - 'code': Unparsed code blocks (custom parsers decompose into above)
|
||
* - 'structural': Boilerplate — keywords, operators, punctuation, formatting
|
||
*
|
||
* Built-in extractors produce: 'title', 'content', 'code'.
|
||
* All 6 categories are available for custom parsers.
|
||
*/
|
||
export type ContentCategory = 'title' | 'annotation' | 'content' | 'value' | 'code' | 'structural'
|
||
|
||
/**
|
||
* A segment of extracted text with its content category
|
||
*
|
||
*/
|
||
export interface ExtractedSegment {
|
||
/** The extracted text content */
|
||
text: string
|
||
/** What role this text plays: 'title' | 'annotation' | 'content' | 'value' | 'code' | 'structural' */
|
||
contentCategory: ContentCategory
|
||
}
|
||
|
||
// ============= Semantic Highlighting Types =============
|
||
|
||
/**
|
||
* Parameters for hybrid highlighting
|
||
*
|
||
* Zero-config highlighting that returns both text (exact) and semantic (concept) matches.
|
||
* Perfect for UI highlighting at different levels.
|
||
*
|
||
* Added contentType hint and contentExtractor callback for structured text
|
||
* (rich-text JSON, HTML, Markdown). Auto-detects format when not specified.
|
||
*
|
||
* @example
|
||
* ```typescript
|
||
* // Plain text
|
||
* const highlights = await brain.highlight({
|
||
* query: "david the warrior",
|
||
* text: "David Smith is a brave fighter who battles dragons"
|
||
* })
|
||
*
|
||
* // Rich-text JSON (auto-detected)
|
||
* const highlights = await brain.highlight({
|
||
* query: "david the warrior",
|
||
* text: JSON.stringify(tiptapDocument)
|
||
* })
|
||
*
|
||
* // Custom extractor (for proprietary formats)
|
||
* const highlights = await brain.highlight({
|
||
* query: "function",
|
||
* text: sourceCode,
|
||
* contentExtractor: (text) => treeSitterParse(text)
|
||
* })
|
||
* ```
|
||
*/
|
||
export interface HighlightParams {
|
||
/** The search query to match against */
|
||
query: string
|
||
|
||
/** The text to highlight (e.g., entity.data) */
|
||
text: string
|
||
|
||
/** Granularity of highlighting: 'word' (default), 'phrase', or 'sentence' */
|
||
granularity?: 'word' | 'phrase' | 'sentence'
|
||
|
||
/** Minimum semantic similarity score for semantic matches (default: 0.5) */
|
||
threshold?: number
|
||
|
||
/**
|
||
* Optional content type hint to skip auto-detection.
|
||
* When omitted, the content type is detected from the text content.
|
||
*/
|
||
contentType?: ContentType
|
||
|
||
/**
|
||
* Optional custom content extractor function.
|
||
* When provided, bypasses built-in detection and extraction entirely.
|
||
* Use this to plug in custom parsers (tree-sitter, Monaco, proprietary formats).
|
||
*/
|
||
contentExtractor?: (text: string) => ExtractedSegment[]
|
||
}
|
||
|
||
/**
|
||
* A highlight showing which text matched the query
|
||
*
|
||
* matchType tells the UI how to style the highlight:
|
||
* - 'text': Exact word match (strongest signal, highest confidence)
|
||
* - 'semantic': Conceptually similar match (may need softer highlight)
|
||
*/
|
||
export interface Highlight {
|
||
/** The text that matched */
|
||
text: string
|
||
|
||
/** Match score (0-1). For text matches, always 1.0. For semantic, varies by similarity. */
|
||
score: number
|
||
|
||
/** Position in original text [start, end] */
|
||
position: [number, number]
|
||
|
||
/** Match type: 'text' (exact word match) or 'semantic' (concept match) */
|
||
matchType: 'text' | 'semantic'
|
||
|
||
/**
|
||
* Content category of the source segment: 'title' | 'annotation' | 'content' | 'value' | 'code' | 'structural'.
|
||
* Present when the input text was structured (JSON, HTML, Markdown).
|
||
*/
|
||
contentCategory?: ContentCategory
|
||
}
|
||
|
||
// ============= Read barrier (waitForIndexed) =============
|
||
|
||
/**
|
||
* One projection leg of the read barrier (`brain.waitForIndexed(path)`) — a
|
||
* derived view of the committed data that queries are served from:
|
||
*
|
||
* - `'semantic'` — the vector index (deferred embeds land here asynchronously)
|
||
* - `'metadata'` — the field/filter index behind `find({ where })`
|
||
* - `'graph'` — the relationship adjacency index
|
||
* - `'aggregation'` — the incremental aggregate states
|
||
*/
|
||
export type IndexedProjectionPath = 'semantic' | 'metadata' | 'graph' | 'aggregation'
|
||
|
||
/**
|
||
* Options for `brain.waitForIndexed()`.
|
||
*/
|
||
export interface WaitForIndexedOptions {
|
||
/**
|
||
* Resolve as soon as the projection has caught up to this committed
|
||
* generation (rather than the current head). Today the pending-embed set
|
||
* carries no generation stamps, so the refinement is conservative: an
|
||
* empty backlog resolves immediately (the watermark is at the head, hence
|
||
* ≥ any committed generation); a non-empty backlog waits for the full
|
||
* drain — a SUPERSET of the requested wait, never a partial one.
|
||
*/
|
||
generation?: number
|
||
|
||
/**
|
||
* Upper bound on the wait in milliseconds. On expiry the promise REJECTS
|
||
* with {@link WaitForIndexedTimeoutError} (typed: the leg + the
|
||
* still-pending count) — never a silent partial wait.
|
||
*/
|
||
timeoutMs?: number
|
||
}
|
||
|
||
/**
|
||
* The typed rejection of `brain.waitForIndexed(path, { timeoutMs })` on
|
||
* expiry. Carries the projection leg (`path`; `'all'` for the no-argument
|
||
* barrier) and the deferred-embed backlog size at the moment the timer fired
|
||
* (`pendingEmbeds` — the same number as
|
||
* `getIndexStatus().projections.semantic.pendingEmbeds`), so a caller can
|
||
* log an honest gauge and retry instead of guessing. A timeout means the
|
||
* projection has NOT caught up — nothing was skipped, nothing partially
|
||
* waited.
|
||
*/
|
||
export class WaitForIndexedTimeoutError extends Error {
|
||
/** The projection leg that had not caught up (`'all'` = the no-arg barrier). */
|
||
public readonly path: IndexedProjectionPath | 'all'
|
||
|
||
/** The expired timeout, in milliseconds. */
|
||
public readonly timeoutMs: number
|
||
|
||
/** Deferred embeds still pending when the timer fired — the live value of
|
||
* `getIndexStatus().projections.semantic.pendingEmbeds`. */
|
||
public readonly pendingEmbeds: number
|
||
|
||
constructor(path: IndexedProjectionPath | 'all', timeoutMs: number, pendingEmbeds: number) {
|
||
super(
|
||
`waitForIndexed(${path === 'all' ? '' : `'${path}'`}) timed out after ${timeoutMs}ms — ` +
|
||
`${pendingEmbeds} deferred embed${pendingEmbeds === 1 ? '' : 's'} still pending; the projection has ` +
|
||
`NOT caught up. Check getIndexStatus().projections.semantic.pendingEmbeds, then retry with a ` +
|
||
`larger timeoutMs or use awaitPendingEmbeds() for an unbounded drain.`
|
||
)
|
||
this.name = 'WaitForIndexedTimeoutError'
|
||
this.path = path
|
||
this.timeoutMs = timeoutMs
|
||
this.pendingEmbeds = pendingEmbeds
|
||
if (Error.captureStackTrace) {
|
||
Error.captureStackTrace(this, WaitForIndexedTimeoutError)
|
||
}
|
||
}
|
||
}
|
||
|
||
// ============= Export all types =============
|
||
|
||
export * from './graphTypes.js' // Re-export NounType, VerbType, etc.
|