feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
/**
|
|
|
|
|
* Brainy Plugin System
|
|
|
|
|
*
|
|
|
|
|
* Simple plugin architecture for two use cases:
|
2026-02-01 08:22:07 -08:00
|
|
|
* 1. Native acceleration (@soulcraft/cortex)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
* 2. Custom storage adapters (e.g., Redis, DynamoDB, custom backends)
|
|
|
|
|
*
|
2026-06-11 14:51:00 -07:00
|
|
|
* Plugins are loaded from an explicit `plugins: [...]` config list or
|
|
|
|
|
* registered manually via `brain.use()` — there is no implicit detection.
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
*/
|
|
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
import type {
|
|
|
|
|
StorageAdapter,
|
|
|
|
|
Vector,
|
|
|
|
|
VectorDocument,
|
|
|
|
|
GraphVerb,
|
|
|
|
|
} from './coreTypes.js'
|
|
|
|
|
import type { NounType, VerbType } from './types/graphTypes.js'
|
|
|
|
|
import type { MetadataIndexStats } from './utils/metadataIndex.js'
|
|
|
|
|
import type { GraphIndexStats } from './graph/graphAdjacencyIndex.js'
|
|
|
|
|
|
|
|
|
|
// Re-export the provider contracts that already live closer to their
|
|
|
|
|
// implementations so a plugin author (Cortex) can import the *entire*
|
|
|
|
|
// provider surface from one stable entrypoint: `@soulcraft/brainy/plugin`.
|
|
|
|
|
export type { ColumnStoreProvider } from './indexes/columnStore/types.js'
|
|
|
|
|
export type {
|
|
|
|
|
AggregationProvider,
|
|
|
|
|
AggregateDefinition,
|
|
|
|
|
AggregateGroupState,
|
|
|
|
|
AggregateResult,
|
|
|
|
|
AggregateQueryParams,
|
|
|
|
|
GroupByDimension,
|
|
|
|
|
} from './types/brainy.types.js'
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Plugin interface — all brainy plugins must implement this.
|
|
|
|
|
*/
|
|
|
|
|
export interface BrainyPlugin {
|
|
|
|
|
/** Unique plugin name (typically the npm package name) */
|
|
|
|
|
name: string
|
|
|
|
|
|
2026-06-19 10:13:04 -07:00
|
|
|
/**
|
|
|
|
|
* Optional semver range of `@soulcraft/brainy` this plugin supports
|
|
|
|
|
* (e.g. `'>=8.0.0 <9.0.0'` or `'^8.0.0'`). When set and the running brainy is
|
|
|
|
|
* OUTSIDE the range, `init()` THROWS rather than silently falling back to the
|
|
|
|
|
* default JS engine. This is the version-coupling guard for the native
|
|
|
|
|
* accelerator (`@soulcraft/cor`): brainy 8.x and cor 3.x are a matched pair,
|
|
|
|
|
* and a mismatch must fail loud, not degrade invisibly. Supported range
|
|
|
|
|
* grammar: space-separated AND of `>=`, `>`, `<=`, `<`, `=` comparators, plus
|
|
|
|
|
* `^X.Y.Z` (same-major floor). Prerelease tags on the running version are
|
|
|
|
|
* tolerated (`8.0.0-rc1` satisfies `>=8.0.0`).
|
|
|
|
|
*/
|
|
|
|
|
brainyRange?: string
|
|
|
|
|
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
/**
|
|
|
|
|
* Called by brainy during init() to activate the plugin.
|
2026-06-19 10:13:04 -07:00
|
|
|
* Return `true` if activation succeeded. Returning `false` is a graceful
|
|
|
|
|
* decline (brainy logs a loud warning and uses the default engine for this
|
|
|
|
|
* plugin's providers). THROWING aborts init — a registered plugin is always
|
|
|
|
|
* explicitly requested (via `config.plugins` or `brain.use()`; brainy does no
|
|
|
|
|
* auto-detection), so a hard failure is fatal, never a silent skip.
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
*/
|
|
|
|
|
activate(context: BrainyPluginContext): Promise<boolean>
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Called when brainy.close() is invoked. Optional cleanup.
|
|
|
|
|
*/
|
|
|
|
|
deactivate?(): Promise<void>
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Context passed to plugins during activation.
|
|
|
|
|
*/
|
|
|
|
|
export interface BrainyPluginContext {
|
|
|
|
|
/**
|
|
|
|
|
* Register a provider for a named subsystem.
|
|
|
|
|
*
|
|
|
|
|
* Well-known provider keys (used by cortex):
|
|
|
|
|
* - 'metadataIndex' — MetadataIndexManager replacement
|
|
|
|
|
* - 'graphIndex' — GraphAdjacencyIndex replacement
|
|
|
|
|
* - 'entityIdMapper' — EntityIdMapper replacement
|
|
|
|
|
* - 'cache' — UnifiedCache replacement
|
2026-06-10 11:29:05 -07:00
|
|
|
* - 'vector' — JsHnswVectorIndex replacement (vector index engine)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
* - 'roaring' — RoaringBitmap32 replacement
|
2026-02-01 13:03:15 -08:00
|
|
|
* - 'embeddings' — Embedding engine replacement (single text)
|
|
|
|
|
* - 'embedBatch' — Batch embedding engine (texts[] → vectors[])
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
* - 'distance' — Distance function overrides
|
|
|
|
|
* - 'msgpack' — Msgpack encode/decode
|
feat: add aggregation engine with incremental SUM/COUNT/AVG/MIN/MAX, GROUP BY, and time windows
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:57:53 -08:00
|
|
|
* - 'aggregation' — AggregationIndex replacement (incremental aggregates)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
*
|
|
|
|
|
* Storage adapter keys:
|
|
|
|
|
* - 'storage:<name>' — Custom storage adapter factory
|
|
|
|
|
*/
|
|
|
|
|
registerProvider(key: string, implementation: unknown): void
|
|
|
|
|
|
|
|
|
|
/** Brainy version for compatibility checks */
|
|
|
|
|
readonly version: string
|
|
|
|
|
}
|
|
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
// ===========================================================================
|
|
|
|
|
// Provider contracts — the EXACT surface Brainy calls on each registered
|
|
|
|
|
// provider.
|
|
|
|
|
//
|
|
|
|
|
// These interfaces are the type-level half of the provider-parity guarantee.
|
|
|
|
|
// They capture only what Brainy actually invokes, so a native accelerator
|
|
|
|
|
// (Cortex) that declares `implements MetadataIndexProvider` gets a compile
|
|
|
|
|
// error the moment a method Brainy depends on is dropped or its signature
|
|
|
|
|
// drifts. Brainy's own baseline classes (`MetadataIndexManager`,
|
2026-06-09 13:07:56 -07:00
|
|
|
// `GraphAdjacencyIndex`, `JsHnswVectorIndex`, `EntityIdMapper`, `UnifiedCache`)
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
// implement them too, so the contract can never silently diverge from the
|
|
|
|
|
// thing Brainy ships.
|
|
|
|
|
//
|
|
|
|
|
// Keep these in lockstep with the call sites in `brainy.ts` and the index
|
|
|
|
|
// classes. When Brainy starts calling a new member, add it here — every
|
|
|
|
|
// implementation then fails to compile until it provides the member.
|
|
|
|
|
// ===========================================================================
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The `'metadataIndex'` provider — a drop-in for `MetadataIndexManager`.
|
|
|
|
|
* Brainy calls this surface via `this.metadataIndex.*` (see `brainy.ts`) and
|
|
|
|
|
* the transactional add/remove operations.
|
|
|
|
|
*/
|
|
|
|
|
export interface MetadataIndexProvider {
|
|
|
|
|
init(): Promise<void>
|
|
|
|
|
flush(): Promise<void>
|
|
|
|
|
rebuild(): Promise<void>
|
|
|
|
|
|
2026-06-30 13:37:23 -07:00
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL. A native provider returns true from the moment its
|
|
|
|
|
* `init()` detects a large epoch-drift until its background
|
|
|
|
|
* build-new→verify→swap has verified-and-swapped. While true, brainy SKIPS its
|
|
|
|
|
* own rebuild for this provider and lets the provider's non-blocking background
|
|
|
|
|
* migration own the index (the no-freeze path); the provider serves correct
|
|
|
|
|
* reads from canonical meanwhile. Mirrors {@link MetadataIndexProvider.init}
|
|
|
|
|
* (and `isReady?()` on the graph provider).
|
|
|
|
|
*/
|
|
|
|
|
isMigrating?(): boolean
|
|
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
addToIndex(id: string, entityOrMetadata: any, skipFlush?: boolean, deferWrites?: boolean): Promise<void>
|
|
|
|
|
removeFromIndex(id: string, metadata?: any): Promise<void>
|
|
|
|
|
|
|
|
|
|
getIds(field: string, value: any): Promise<string[]>
|
feat(8.0): wire the two cor-confirmed metadata-provider contract additions
Both shapes confirmed with cor for the lockstep (cor builds the native side
against these; the JS index is a no-op / unchanged):
- probeConsistency?(): Promise<boolean> — OPTIONAL O(1) cold-open consistency
sampler on MetadataIndexProvider. brainy calls it ONCE per brain on the first
read; on `false` it self-heals via detectAndRepairCorruption() — the metadata
counterpart of the 7.33.2 graph cold-load guard, completing the phantom triad's
detect-and-repair-on-open. Best-effort: a probe failure never breaks a read
(the one-shot guard resets so a transient failure retries). The JS index omits
the method, so it is simply never probed.
- getIdsForFilter(filter, opts?: { limit?; offset? }) — brainy passes a page
bound on the UNSORTED find({ type, where, limit }) path so a native provider can
early-stop and return only the [0, limit) prefix, killing the O(N) FFI marshal
at billion scale (pairs with cor #75). brainy passes offset:0 and ALWAYS
re-windows the result itself (visibility filter + slice); the JS index ignores
opts and returns all matches (behaviour unchanged).
New unit tests cover brainy's call behaviour for both (probe→repair on false,
healthy→no-repair, failure-is-best-effort, once-per-brain; opts passed with
offset 0 on the unsorted path). Full gate green: unit 1718, integration 607.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:11:00 -07:00
|
|
|
/**
|
|
|
|
|
* Resolve a `where` filter to its matching ids.
|
|
|
|
|
* @param filter - The filter shape (eq / allOf / anyOf / ne / range / exists / …).
|
|
|
|
|
* @param opts - OPTIONAL page bound for the UNSORTED `find({ type, where, limit })`
|
|
|
|
|
* path. Brainy passes `{ limit: offset+limit (+hidden over-fetch), offset: 0 }` and
|
|
|
|
|
* ALWAYS re-windows the result itself (visibility filter + `slice`), so a provider
|
|
|
|
|
* that honors `opts` should early-stop and return the `[0, limit)` PREFIX (it must
|
|
|
|
|
* NOT pre-apply `offset`). The JS index ignores `opts` and returns all matches —
|
|
|
|
|
* so honoring it is a pure native-side optimization that removes the O(N) FFI
|
|
|
|
|
* marshal at billion scale. Pairs with {@link getIdSetForFilter}.
|
|
|
|
|
*/
|
|
|
|
|
getIdsForFilter(filter: any, opts?: { limit?: number; offset?: number }): Promise<string[]>
|
2026-06-23 13:18:34 -07:00
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL: resolve a `where` filter to its matching id universe as
|
|
|
|
|
* an {@link OpaqueIdSet} (a serialized roaring `Buffer`) WITHOUT materializing
|
|
|
|
|
* the ids in TypeScript — the producer half of the predicate-pushdown
|
|
|
|
|
* (`CTX-PUSHDOWN-ALLOWEDIDS`) and graph query→expand fusion. Brainy forwards the
|
|
|
|
|
* returned set opaquely to `VectorIndexProvider.search({ allowedIds })` and
|
|
|
|
|
* `GraphAccelerationProvider.traverse(seeds)`; it NEVER inspects it. A native
|
|
|
|
|
* (cor) metadata index returns its roaring filter result directly (zero
|
|
|
|
|
* crossing); the JS index does not implement this — Brainy falls back to
|
|
|
|
|
* {@link MetadataIndexProvider.getIdsForFilter} + a `ReadonlySet<string>` when
|
|
|
|
|
* absent. Present ⟺ the native stack is the active provider, so the matching
|
|
|
|
|
* native vector/graph engines can decode the same envelope.
|
|
|
|
|
* @param filter - The same filter shape accepted by `getIdsForFilter`.
|
|
|
|
|
* @returns The matching id universe as an opaque set.
|
|
|
|
|
*/
|
|
|
|
|
getIdSetForFilter?(filter: any): Promise<OpaqueIdSet>
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
getIdsForTextQuery(query: string): Promise<Array<{ id: string; matchCount: number }>>
|
2026-06-23 15:01:49 -07:00
|
|
|
getSortedIdsForFilter(filter: any, orderBy: string, order?: 'asc' | 'desc', topK?: number): Promise<string[]>
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
getFilterValues(field: string): Promise<string[]>
|
|
|
|
|
getFilterFields(): Promise<string[]>
|
|
|
|
|
getFieldValueForEntity(entityId: string, field: string): Promise<any>
|
|
|
|
|
getFieldsForType(nounType: NounType): Promise<Array<{ field: string; affinity: number; occurrences: number; totalEntities: number }>>
|
|
|
|
|
getFieldStatistics(): Promise<Map<string, unknown>>
|
|
|
|
|
getFieldsWithCardinality(): Promise<Array<{ field: string; cardinality: number; distribution: string }>>
|
|
|
|
|
getOptimalQueryPlan(filters: Record<string, any>): Promise<unknown>
|
|
|
|
|
|
|
|
|
|
/** Report which index path a `where` clause on `field` will hit (drives `brain.explain()`). */
|
|
|
|
|
explainField(field: string): Promise<{ path: 'column-store' | 'sparse-chunked' | 'none'; notes?: string }>
|
|
|
|
|
|
|
|
|
|
getCountForCriteria(field: string, value: any): Promise<number>
|
|
|
|
|
getEntityCountByType(type: string): number
|
|
|
|
|
getEntityCountByTypeEnum(type: NounType): number
|
|
|
|
|
getTotalEntityCount(): number
|
|
|
|
|
getAllEntityCounts(): Map<string, number>
|
|
|
|
|
getTopNounTypes(n: number): NounType[]
|
|
|
|
|
getTopVerbTypes(n: number): VerbType[]
|
|
|
|
|
getAllNounTypeCounts(): Map<NounType, number>
|
|
|
|
|
getAllVerbTypeCounts(): Map<VerbType, number>
|
|
|
|
|
getAllVFSEntityCounts(): Promise<Map<string, number>>
|
|
|
|
|
|
|
|
|
|
detectAndRepairCorruption(): Promise<void>
|
feat(8.0): wire the two cor-confirmed metadata-provider contract additions
Both shapes confirmed with cor for the lockstep (cor builds the native side
against these; the JS index is a no-op / unchanged):
- probeConsistency?(): Promise<boolean> — OPTIONAL O(1) cold-open consistency
sampler on MetadataIndexProvider. brainy calls it ONCE per brain on the first
read; on `false` it self-heals via detectAndRepairCorruption() — the metadata
counterpart of the 7.33.2 graph cold-load guard, completing the phantom triad's
detect-and-repair-on-open. Best-effort: a probe failure never breaks a read
(the one-shot guard resets so a transient failure retries). The JS index omits
the method, so it is simply never probed.
- getIdsForFilter(filter, opts?: { limit?; offset? }) — brainy passes a page
bound on the UNSORTED find({ type, where, limit }) path so a native provider can
early-stop and return only the [0, limit) prefix, killing the O(N) FFI marshal
at billion scale (pairs with cor #75). brainy passes offset:0 and ALWAYS
re-windows the result itself (visibility filter + slice); the JS index ignores
opts and returns all matches (behaviour unchanged).
New unit tests cover brainy's call behaviour for both (probe→repair on false,
healthy→no-repair, failure-is-best-effort, once-per-brain; opts passed with
offset 0 on the unsorted path). Full gate green: unit 1718, integration 607.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:11:00 -07:00
|
|
|
/**
|
|
|
|
|
* OPTIONAL cheap (O(1)) cold-open consistency probe — the counterpart of the
|
|
|
|
|
* graph cold-load guard. Returns `true` if a sampled `(field, value, int)` is
|
|
|
|
|
* clean, `false` if it detects the cross-bucket phantom signature (an int whose
|
|
|
|
|
* current value for `field` no longer equals `value`). Brainy calls it ONCE per
|
|
|
|
|
* brain on the first read and, on `false`, runs {@link detectAndRepairCorruption}
|
|
|
|
|
* to self-heal — so an already-poisoned index repairs itself on open without the
|
|
|
|
|
* cost of the full-scan {@link validateConsistency}. A provider that omits it is
|
|
|
|
|
* simply never auto-probed (no behavior change).
|
|
|
|
|
*/
|
|
|
|
|
probeConsistency?(): Promise<boolean>
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
validateConsistency(): Promise<{
|
|
|
|
|
healthy: boolean
|
|
|
|
|
avgEntriesPerEntity: number
|
|
|
|
|
entityCount: number
|
|
|
|
|
indexEntryCount: number
|
|
|
|
|
recommendation: string | null
|
|
|
|
|
}>
|
|
|
|
|
|
|
|
|
|
tokenize(text: string): string[]
|
|
|
|
|
extractTextContent(data: any): string
|
|
|
|
|
|
|
|
|
|
getStats(): Promise<MetadataIndexStats>
|
|
|
|
|
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
/**
|
|
|
|
|
* The shared UUID ↔ int mapper — the single source of truth for entity-int
|
|
|
|
|
* resolution at the provider boundary. The coordinator (`brainy.ts`) resolves
|
|
|
|
|
* UUID → int exactly once before every graph-index call (`getOrAssign` on
|
|
|
|
|
* writes, `getInt` on reads — `undefined` means "never mapped", i.e. the
|
|
|
|
|
* entity has no relations) and converts provider-returned ints back with
|
|
|
|
|
* `getUuid`. Ints are u32 today (the `EntityIdSpaceExceeded` guard enforces
|
|
|
|
|
* the ceiling on the JS path), so `Number(bigint)` narrowing is lossless.
|
|
|
|
|
*/
|
|
|
|
|
getIdMapper(): {
|
|
|
|
|
getOrAssign(uuid: string): number
|
|
|
|
|
getInt(uuid: string): number | undefined
|
|
|
|
|
getUuid(intId: number): string | undefined
|
|
|
|
|
}
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
|
|
|
|
|
/** The column store the coordinator delegates `where`/`orderBy` to. */
|
|
|
|
|
readonly columnStore: import('./indexes/columnStore/types.js').ColumnStoreProvider
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The `'graphIndex'` provider — a drop-in for `GraphAdjacencyIndex`.
|
|
|
|
|
* Brainy calls this surface via `this.graphIndex.*` (some optional-chained).
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
*
|
|
|
|
|
* **8.0 u64 contract — BigInt at the boundary.** Reads take entity ints
|
|
|
|
|
* (from the metadata index's idMapper) and return entity/verb ints as
|
|
|
|
|
* `bigint[]`. The coordinator owns ALL UUID ↔ int conversion: it resolves
|
|
|
|
|
* UUIDs to ints once at the `brainy.ts` boundary (`getOrAssign` on writes,
|
|
|
|
|
* `getInt` on reads, returning empty results for unmapped UUIDs without
|
|
|
|
|
* calling the provider) and maps returned ints back (`getUuid` for entities,
|
|
|
|
|
* {@link GraphIndexProvider.verbIntsToIds} for verbs). Implementations may
|
|
|
|
|
* stay u32 internally — `Number(bigint)` narrowing is lossless under the
|
|
|
|
|
* shipped `EntityIdSpaceExceeded` u32 guard — but must speak the bigint
|
|
|
|
|
* contract at this surface.
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
*/
|
|
|
|
|
export interface GraphIndexProvider {
|
2026-06-29 10:04:19 -07:00
|
|
|
/**
|
|
|
|
|
* `false` until the provider has loaded its persisted state. A provider should
|
|
|
|
|
* self-load on first read; this flag lets it report readiness for diagnostics
|
|
|
|
|
* and tooling. (Brainy's graph reads route through the provider's own methods,
|
|
|
|
|
* which are expected to be self-loading — it is the {@link GraphAccelerationProvider}
|
|
|
|
|
* fast paths that gate on `isInitialized`.)
|
|
|
|
|
*/
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
readonly isInitialized: boolean
|
|
|
|
|
|
fix(8.0): never serve a silent [] from find({connected}) on a cold-loaded graph
8.0 shipped with no cold-graph guard (the 7.33.4 fix was never ported), so on a
cold open of a large brain where the native adjacency reports membership but its
source->target edges did not load, find({connected})/neighbors()/related() could
silently return [] for persisted edges.
Add the converged isReady() contract: GraphIndexProvider.isReady?(): boolean is
the honest cold-load readiness signal (true ONLY when edges are loaded), so brainy
gates on it instead of the lying membership-size() proxy. verifyGraphAdjacencyLive()
checks it — Strategy 1: false -> hydrate the id-mapper, rebuild from storage, re-check,
throw GraphIndexNotReadyError if still not ready; providers without isReady() fall
back to a global known-edge-sample probe (Strategy 2, not the queried anchor, so a
genuinely edgeless node still returns []). rebuildIndexesIfNeeded gates the graph
rebuild on it too, with the id-mapper hydrated before the adjacency rebuild on the
lazy cold-open path (the CTX-BR-RESTORE-REBUILD ordering, shared with restore()).
executeGraphSearch re-verifies before trusting an empty connected result and
re-collects on a heal. 6-case integration test + 1718 unit green.
2026-06-30 09:30:11 -07:00
|
|
|
/**
|
|
|
|
|
* @description Returns true ONLY when the source→target EDGES are loaded (NOT
|
|
|
|
|
* membership/manifest count) — the honest cold-load readiness signal. brainy
|
|
|
|
|
* gates the graph rebuild + the connected read-time guard on it: a `false`
|
|
|
|
|
* forces a rebuild from storage, and a still-`false` after that rebuild raises
|
|
|
|
|
* a loud {@link import('./errors/brainyError.js').GraphIndexNotReadyError}
|
|
|
|
|
* rather than serving an empty traversal as truth. cortex >= 2.7.8 (2.x) / 3.0
|
|
|
|
|
* exposes it; ABSENT on older providers, where brainy falls back to a
|
|
|
|
|
* known-edge-sample probe.
|
|
|
|
|
* @returns `true` when the persisted adjacency is loaded and traversals are
|
|
|
|
|
* trustworthy; `false` when only the count/manifest loaded (cold-open).
|
|
|
|
|
*/
|
|
|
|
|
isReady?(): boolean
|
|
|
|
|
|
2026-06-30 09:55:19 -07:00
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL eager cold-load. Called once during brain init — AFTER
|
|
|
|
|
* the metadata provider's `init()` (so the id-mapper is hydrated; a native int
|
|
|
|
|
* adjacency resolves endpoints through it — the id-mapper-before-adjacency
|
|
|
|
|
* order) and BEFORE the rebuild gate — so a native provider loads its
|
|
|
|
|
* source→target adjacency from storage and reports `isReady() === true` at the
|
|
|
|
|
* gate, with no spurious rebuild (the §7.1 `rebuild()==0` acceptance). Mirrors
|
|
|
|
|
* {@link MetadataIndexProvider.init}. The JS graph index omits it and
|
|
|
|
|
* self-loads its adjacency on demand.
|
|
|
|
|
*/
|
|
|
|
|
init?(): Promise<void>
|
|
|
|
|
|
2026-06-30 13:37:23 -07:00
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL. A native provider returns true from the moment its
|
|
|
|
|
* `init()` detects a large epoch-drift until its background
|
|
|
|
|
* build-new→verify→swap has verified-and-swapped. While true, brainy SKIPS its
|
|
|
|
|
* own rebuild for this provider and lets the provider's non-blocking background
|
|
|
|
|
* migration own the index (the no-freeze path); the provider serves correct
|
|
|
|
|
* reads from canonical meanwhile. Mirrors `isReady?()` / `init?()`.
|
|
|
|
|
*/
|
|
|
|
|
isMigrating?(): boolean
|
|
|
|
|
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
/**
|
|
|
|
|
* @description Entity ints reachable from `id` (1 hop), deduped.
|
|
|
|
|
* @param id - The entity's interned int (from the shared idMapper).
|
|
|
|
|
* @param options - Direction (`'both'` default) and limit/offset pagination.
|
|
|
|
|
* @returns Neighbor entity ints. Empty when the entity has no edges.
|
|
|
|
|
*/
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
getNeighbors(
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
id: bigint,
|
|
|
|
|
options?: { direction?: 'in' | 'out' | 'both'; limit?: number; offset?: number }
|
|
|
|
|
): Promise<bigint[]>
|
|
|
|
|
/**
|
|
|
|
|
* @description Verb ints for all edges originating at `sourceInt`.
|
|
|
|
|
* @param sourceInt - The source entity's interned int.
|
|
|
|
|
* @param options - Optional limit/offset pagination.
|
|
|
|
|
* @returns Verb ints, resolvable via {@link GraphIndexProvider.verbIntsToIds}.
|
|
|
|
|
*/
|
|
|
|
|
getVerbIdsBySource(sourceInt: bigint, options?: { limit?: number; offset?: number }): Promise<bigint[]>
|
|
|
|
|
/**
|
|
|
|
|
* @description Verb ints for all edges pointing at `targetInt`.
|
|
|
|
|
* @param targetInt - The target entity's interned int.
|
|
|
|
|
* @param options - Optional limit/offset pagination.
|
|
|
|
|
* @returns Verb ints, resolvable via {@link GraphIndexProvider.verbIntsToIds}.
|
|
|
|
|
*/
|
|
|
|
|
getVerbIdsByTarget(targetInt: bigint, options?: { limit?: number; offset?: number }): Promise<bigint[]>
|
|
|
|
|
/**
|
|
|
|
|
* @description Batch reverse resolver: verb ints → verb-id strings. REQUIRED —
|
|
|
|
|
* the provider owns the durable verb-int interning (Brainy keeps only a
|
|
|
|
|
* bounded in-memory warm cache fed by `addVerb` returns and this resolver;
|
|
|
|
|
* pure optimization, no durability role).
|
|
|
|
|
* @param verbInts - Verb ints as returned by the read methods.
|
|
|
|
|
* @returns One entry per input, order-preserving; `null` for unknown ints.
|
|
|
|
|
*/
|
|
|
|
|
verbIntsToIds(verbInts: bigint[]): Promise<(string | null)[]>
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
getVerbsBatchCached(verbIds: string[]): Promise<Map<string, GraphVerb>>
|
|
|
|
|
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
/**
|
|
|
|
|
* @description Index one verb. The coordinator resolves both endpoint ints
|
|
|
|
|
* via `idMapper.getOrAssign` and mirrors them onto `verb.sourceInt` /
|
|
|
|
|
* `verb.targetInt` before the call.
|
|
|
|
|
* @param verb - The verb to index (endpoint UUIDs + derived `sourceInt`/`targetInt`).
|
|
|
|
|
* @param sourceInt - The source entity's interned int.
|
|
|
|
|
* @param targetInt - The target entity's interned int.
|
2026-06-16 09:41:59 -07:00
|
|
|
* @param generation - Brainy's commit generation for this write — the same
|
|
|
|
|
* watermark the storage layer stamps onto the record. A provider with a
|
|
|
|
|
* per-generation edge chain records the edge's existence at this generation
|
|
|
|
|
* so `db.asOf(g)` graph hops resolve historically correct endpoints; the
|
|
|
|
|
* JS baseline has no such chain and ignores it (graph time-travel is a
|
|
|
|
|
* native-provider capability — see the consistency-model doc).
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
* @returns The interned verb int for `verb.id` (feeds Brainy's warm cache).
|
|
|
|
|
*/
|
2026-06-16 09:41:59 -07:00
|
|
|
addVerb(verb: GraphVerb, sourceInt: bigint, targetInt: bigint, generation: bigint): Promise<bigint>
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
/**
|
|
|
|
|
* @description Remove one verb from the index by its id string. The verb's
|
|
|
|
|
* interned int stays reserved (ints are never recycled within a generation).
|
|
|
|
|
* @param verbId - The verb's UUID string.
|
2026-06-16 09:41:59 -07:00
|
|
|
* @param generation - Brainy's commit generation for this removal. A provider
|
|
|
|
|
* with a per-generation edge chain tombstones the edge at this generation
|
|
|
|
|
* (so it remains visible to `db.asOf(g)` for `g` before the removal); the
|
|
|
|
|
* JS baseline removes immediately and ignores it.
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
* @returns Resolves once the verb no longer appears in reads.
|
|
|
|
|
*/
|
2026-06-16 09:41:59 -07:00
|
|
|
removeVerb(verbId: string, generation: bigint): Promise<void>
|
feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror):
- getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and
return entity/verb ints as bigint[]
- new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7
identity-fingerprint design — verb ids are UUIDs by contract, so the
provider-side interning is losslessly reversible)
- addVerb(verb, sourceInt, targetInt) returns the interned verb int;
removeVerb(verbId) joins the contract
Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on
writes, getInt on reads (unmapped UUID -> empty result without calling the
provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry
insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb
returns and resolver results. GraphVerb gains derived sourceInt/targetInt
(populated at add time, never persisted). findConnectedSubtype gains a
native fast path that routes single-type single-subtype outgoing BFS
through the provider when available.
JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed
internally: entity ints resolve through the shared entity-id mapper (threaded
in by the coordinator on init/fork/checkout), verb ints come from an
in-process append-only interning map re-derived from storage on
rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/
removeEntity take bigint, sortTopK/filteredSortTopK return bigint[].
relate() now rejects a caller-supplied id with a teaching error — verb ids
are brainy-generated UUIDs by contract in 8.0 (previously a passed id was
silently ignored). No Roaring64 provider-boundary decode site exists yet;
the JS-internal column store stays Roaring32 and the Treemap decoder lands
with the first consumer of provider-returned filter buffers.
Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
2026-06-10 10:45:45 -07:00
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
rebuild(): Promise<void>
|
|
|
|
|
flush(): Promise<void>
|
|
|
|
|
close(): Promise<void>
|
|
|
|
|
size(): number
|
|
|
|
|
|
|
|
|
|
getStats(): GraphIndexStats
|
|
|
|
|
getRelationshipStats(): {
|
|
|
|
|
totalRelationships: number
|
|
|
|
|
relationshipsByType: Record<string, number>
|
|
|
|
|
uniqueSourceNodes: number
|
|
|
|
|
uniqueTargetNodes: number
|
|
|
|
|
totalNodes: number
|
|
|
|
|
}
|
|
|
|
|
getRelationshipCountByType(type: string): number
|
|
|
|
|
getTotalRelationshipCount(): number
|
|
|
|
|
getAllRelationshipCounts(): Map<string, number>
|
|
|
|
|
}
|
|
|
|
|
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
// ============= Graph Acceleration (optional native engine) =============
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Opaque serialized id-set — a cor-internal roaring payload (a Node `Buffer` at
|
|
|
|
|
* runtime when the native engine is present), passed straight from a `find()`
|
|
|
|
|
* universe into `traverse` / `VectorIndexProvider.search` with NO id
|
|
|
|
|
* materialization in TypeScript (the O(1)-crossing query→expand win). Brainy
|
|
|
|
|
* NEVER inspects it; the provider version-tags the envelope and throws on a
|
|
|
|
|
* format mismatch (it owns integrity + the id-space-width guarantee). Typed as
|
|
|
|
|
* `Uint8Array` (which a Node `Buffer` satisfies) so the universal build stays
|
|
|
|
|
* browser-safe.
|
|
|
|
|
*/
|
|
|
|
|
export type OpaqueIdSet = Uint8Array
|
|
|
|
|
|
|
|
|
|
/** Direction of a graph traversal relative to each frontier node. */
|
|
|
|
|
export type GraphTraversalDirection = 'in' | 'out' | 'both'
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Columnar subgraph — the shared wire format for every native graph read
|
|
|
|
|
* (`traverse`, `edgesForNode`, cursor chunks). Parallel typed arrays, never an
|
|
|
|
|
* array-of-objects, so the NAPI boundary transfers them near-zero-copy and
|
|
|
|
|
* Brainy maps u64↔UUID **lazily** (only for the rows a caller actually renders,
|
|
|
|
|
* via `EntityIdMapperProvider.entityIntsToUuids` + `GraphIndexProvider.verbIntsToIds`).
|
|
|
|
|
* The three `edge*` arrays are parallel (index `i` is one edge); `nodes` /
|
|
|
|
|
* `nodeDepth` are parallel.
|
|
|
|
|
*/
|
|
|
|
|
export interface Subgraph {
|
|
|
|
|
/** Discovered entity ints; resolve via `entityIntsToUuids`. */
|
|
|
|
|
nodes: BigInt64Array
|
|
|
|
|
/** Hop distance of each node from the nearest seed (parallel to `nodes`). Present iff `includeDepth`. */
|
|
|
|
|
nodeDepth?: Uint8Array
|
|
|
|
|
/** Edge source entity ints. */
|
|
|
|
|
edgeSources: BigInt64Array
|
|
|
|
|
/** Edge target entity ints. */
|
|
|
|
|
edgeTargets: BigInt64Array
|
|
|
|
|
/** Edge verb ints; resolve via `verbIntsToIds`. */
|
|
|
|
|
edgeVerbInts: BigInt64Array
|
|
|
|
|
/** Edge verb-type indices (stable TypeIdx; resolve via `TypeUtils.getVerbFromIndex`). */
|
|
|
|
|
edgeTypes: Uint16Array
|
|
|
|
|
/**
|
|
|
|
|
* `true` when a `maxNodes` / `maxEdges` cap truncated the result. The returned
|
|
|
|
|
* rows are the deterministic BFS-order prefix; page the remainder with a
|
|
|
|
|
* `graphCursor` (traverse stays stateless/bounded — no continuation token).
|
|
|
|
|
*/
|
|
|
|
|
truncated?: boolean
|
|
|
|
|
/** When `truncated`, how many nodes/edges were cut (for "+N more" affordances). */
|
|
|
|
|
truncatedNodeCount?: number
|
|
|
|
|
truncatedEdgeCount?: number
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Knobs shared by the bounded traversals. */
|
|
|
|
|
export interface TraverseOptions {
|
|
|
|
|
/** Max hop distance from any seed. */
|
|
|
|
|
depth: number
|
|
|
|
|
/** Edge direction to follow (default `'both'`). */
|
|
|
|
|
direction?: GraphTraversalDirection
|
|
|
|
|
/** Restrict traversed edges to these verb-type indices (TypeIdx). */
|
|
|
|
|
verbTypes?: number[]
|
|
|
|
|
/** Restrict traversed edges to these subtypes. */
|
|
|
|
|
subtypes?: string[]
|
|
|
|
|
/**
|
|
|
|
|
* Visibility tiers to EXCLUDE from the frontier (default: `['internal','system']`,
|
|
|
|
|
* matching `related()`). Pass `[]` for an all-tiers (admin) view.
|
|
|
|
|
*/
|
|
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
/** Cap on returned nodes (deterministic BFS-order prefix; sets `Subgraph.truncated`). */
|
|
|
|
|
maxNodes?: number
|
|
|
|
|
/** Cap on returned edges. */
|
|
|
|
|
maxEdges?: number
|
|
|
|
|
/** Include the edge arrays (`false` = nodes-only reachability). Default `true`. */
|
|
|
|
|
includeEdges?: boolean
|
|
|
|
|
/** Populate `Subgraph.nodeDepth`. Default `false`. */
|
|
|
|
|
includeDepth?: boolean
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Options for the both-direction single-node edge read. */
|
|
|
|
|
export interface EdgesForNodeOptions {
|
|
|
|
|
/** Edge direction (default `'both'`). */
|
|
|
|
|
direction?: GraphTraversalDirection
|
|
|
|
|
verbTypes?: number[]
|
|
|
|
|
subtypes?: string[]
|
|
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
limit?: number
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Opaque server-side cursor handle (TTL-bounded; pinned to a generation at open). */
|
|
|
|
|
export type GraphCursorHandle = string
|
|
|
|
|
|
|
|
|
|
/** Options for opening a streaming whole-graph (or seeded) cursor. */
|
|
|
|
|
export interface GraphCursorOptions {
|
|
|
|
|
direction?: GraphTraversalDirection
|
|
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
/** `'light'` = ids/types only, no metadata-join columns (viz default); `'full'` = with joins. */
|
|
|
|
|
projection?: 'light' | 'full'
|
|
|
|
|
/** Seed set to bound the walk; omitted = the whole graph. */
|
|
|
|
|
seeds?: bigint[] | OpaqueIdSet
|
|
|
|
|
/**
|
|
|
|
|
* Resume token from a prior chunk — used to continue after a `SnapshotExpired`
|
|
|
|
|
* (the pinned generation's TTL lapsed): reopen with this to resume the walk.
|
|
|
|
|
*/
|
|
|
|
|
cursor?: string
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** One streamed chunk of a graph cursor walk. */
|
|
|
|
|
export interface GraphCursorChunk {
|
|
|
|
|
/** The chunk's nodes + edges, columnar. */
|
|
|
|
|
subgraph: Subgraph
|
|
|
|
|
/** Opaque resume token (pass to `graphCursorOpen({ cursor })` after `SnapshotExpired`). */
|
|
|
|
|
cursor?: string
|
|
|
|
|
/** `true` once the walk is exhausted; `graphCursorNext` should not be called again. */
|
|
|
|
|
done: boolean
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Per-node scores returned by `rank` / `mostConnected`; `scores[i]` belongs to `nodeInts[i]` (descending). */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
export interface GraphScores {
|
|
|
|
|
nodeInts: BigInt64Array
|
|
|
|
|
scores: Float64Array
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Community/cluster labelling; `communityIds[i]` is the group of `nodeInts[i]`. */
|
|
|
|
|
export interface GraphCommunities {
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
nodeInts: BigInt64Array
|
2026-06-22 09:54:01 -07:00
|
|
|
communityIds: Uint32Array
|
|
|
|
|
communityCount: number
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** A path between two nodes — the node sequence + the edges between them (or `null` if unreachable). */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
export interface GraphPath {
|
|
|
|
|
nodeInts: BigInt64Array
|
|
|
|
|
edgeVerbInts: BigInt64Array
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Cost of the path: number of hops, or summed edge weight when `by: 'weight'`. */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
cost: number
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/**
|
|
|
|
|
* Options for `rank` (importance / influence ranking). INTENT-level only — the
|
|
|
|
|
* ranking ALGORITHM is the provider's choice (the JS fallback uses PageRank; a
|
|
|
|
|
* native provider may use personalized PageRank, eigenvector centrality, etc.),
|
|
|
|
|
* so no algorithm-tuning knobs are exposed here.
|
|
|
|
|
*/
|
|
|
|
|
export interface RankOptions {
|
|
|
|
|
/** Return only the top-K nodes by score (default: all). */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
topK?: number
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Visibility tiers to EXCLUDE from the ranked set (default `['internal','system']`). */
|
|
|
|
|
excludeVisibility?: string[]
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Options for `communities` (grouping related nodes). The grouping algorithm is the provider's choice. */
|
|
|
|
|
export interface CommunitiesOptions {
|
|
|
|
|
/** Treat the graph as directed when grouping (default `false` — direction-agnostic). */
|
|
|
|
|
directed?: boolean
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Options for `path` (best route between two nodes). */
|
|
|
|
|
export interface PathOptions {
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
direction?: GraphTraversalDirection
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Optimize for fewest `'hops'` (default) or least summed edge `'weight'`. */
|
|
|
|
|
by?: 'hops' | 'weight'
|
|
|
|
|
/** Restrict the route to these verb-type indices (TypeIdx). */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
verbTypes?: number[]
|
|
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
/** Abandon the search past this many hops. */
|
|
|
|
|
maxDepth?: number
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Options for `sample` (a representative neighborhood sample for dense-graph viz). */
|
|
|
|
|
export interface SampleOptions {
|
|
|
|
|
/** Max hop distance from any seed. */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
depth: number
|
|
|
|
|
direction?: GraphTraversalDirection
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Max neighbors kept per node (random, seeded for reproducibility). */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
fanout: number
|
2026-06-22 09:54:01 -07:00
|
|
|
/** RNG seed so a given (seeds, opts, seed) yields a stable sample. */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
seed?: number
|
|
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
maxNodes?: number
|
|
|
|
|
}
|
|
|
|
|
|
2026-06-22 09:54:01 -07:00
|
|
|
/** Options for `mostConnected` (the most-connected nodes). */
|
|
|
|
|
export interface MostConnectedOptions {
|
|
|
|
|
/** Return the top-K most-connected nodes. */
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
topK: number
|
2026-06-22 09:54:01 -07:00
|
|
|
direction?: GraphTraversalDirection
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
excludeVisibility?: string[]
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The `'graphAcceleration'` provider — an OPTIONAL native graph engine (the
|
|
|
|
|
* cor 3.0 acceleration layer). Brainy feature-detects it on the registered
|
|
|
|
|
* providers: when present, the public `brain.graph.*` surface and the
|
|
|
|
|
* `related({ node })` / `neighbors({ depth })` / `find({ connected })` paths
|
|
|
|
|
* route here; when absent, Brainy serves the same operations from its pure-TS
|
|
|
|
|
* adjacency fallback (correct, small-graph-scale). It is SEPARATE from the
|
|
|
|
|
* required {@link GraphIndexProvider} (which stays the bigint-adjacency
|
|
|
|
|
* contract) so "optional native acceleration" is explicit at the seam.
|
|
|
|
|
*
|
|
|
|
|
* **Generation-aware (8.0 time-travel).** Every read takes an optional trailing
|
|
|
|
|
* `generation?: bigint`; omitted = the current generation ("now"). With a
|
|
|
|
|
* generation, the provider resolves the graph **as of** that generation
|
|
|
|
|
* (`db.asOf(g).graph.*`), reading its generation-filtered adjacency rather than
|
|
|
|
|
* the now-only fast path.
|
|
|
|
|
*
|
|
|
|
|
* **Columnar + opaque-id-set.** Reads return the columnar {@link Subgraph};
|
|
|
|
|
* `traverse` / sample accept seeds as `bigint[]` OR an {@link OpaqueIdSet}
|
|
|
|
|
* (a `find()` universe forwarded with no id materialization — the query→expand
|
|
|
|
|
* fusion). Visibility tiers are excluded at the index by default.
|
|
|
|
|
*/
|
|
|
|
|
export interface GraphAccelerationProvider {
|
|
|
|
|
/** `false` until the native engine has loaded; Brainy probes before routing. */
|
|
|
|
|
readonly isInitialized: boolean
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description Bounded multi-hop expansion from `seeds`, returning the reachable
|
|
|
|
|
* subgraph. One call replaces Brainy's per-hop BFS round-trips.
|
|
|
|
|
* @param seeds - Start nodes as entity ints, OR an {@link OpaqueIdSet} (a
|
|
|
|
|
* `find()` universe — the frontier is intersected in id-space, zero crossing).
|
|
|
|
|
* @param options - Depth, direction, edge/visibility filters, and caps.
|
|
|
|
|
* @param generation - Optional as-of generation (omitted = now).
|
|
|
|
|
* @returns The reachable {@link Subgraph} (BFS-order; `truncated` set if capped).
|
|
|
|
|
*/
|
|
|
|
|
traverse(seeds: bigint[] | OpaqueIdSet, options: TraverseOptions, generation?: bigint): Promise<Subgraph>
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description All edges incident to one node, both directions, as structure +
|
|
|
|
|
* cor-indexed fields (Brainy hydrates full edge metadata lazily — the provider
|
|
|
|
|
* cannot recover verb-id strings from its interned form).
|
|
|
|
|
* @param nodeInt - The node's interned entity int.
|
|
|
|
|
* @param options - Direction, edge/visibility filters, limit.
|
|
|
|
|
* @param generation - Optional as-of generation.
|
|
|
|
|
* @returns A {@link Subgraph} of the node's incident edges.
|
|
|
|
|
*/
|
|
|
|
|
edgesForNode(nodeInt: bigint, options: EdgesForNodeOptions, generation?: bigint): Promise<Subgraph>
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description Open a snapshot-consistent streaming walk of the whole graph (or a
|
|
|
|
|
* seeded region) for O(N) viz loads. Pins a generation at open; the walk reads
|
|
|
|
|
* as-of that pin (no dup/skip under concurrent writes). Handles are TTL-bounded —
|
|
|
|
|
* `graphCursorNext` after expiry rejects with `SnapshotExpired`; reopen with the
|
|
|
|
|
* last chunk's `cursor` to resume. Call `graphCursorClose` on normal completion.
|
|
|
|
|
* @param options - Direction, visibility, projection mode, optional seeds / resume cursor.
|
|
|
|
|
* @param generation - Optional explicit as-of generation to pin (else pins current).
|
|
|
|
|
* @returns A handle for `graphCursorNext` / `graphCursorClose`.
|
|
|
|
|
*/
|
|
|
|
|
graphCursorOpen(options: GraphCursorOptions, generation?: bigint): Promise<GraphCursorHandle>
|
|
|
|
|
/**
|
|
|
|
|
* @description Pull the next chunk from an open cursor.
|
|
|
|
|
* @param handle - From `graphCursorOpen`.
|
|
|
|
|
* @param chunkSize - Target number of nodes/edges in this chunk.
|
|
|
|
|
* @returns The next {@link GraphCursorChunk}; `done: true` when exhausted.
|
|
|
|
|
*/
|
|
|
|
|
graphCursorNext(handle: GraphCursorHandle, chunkSize: number): Promise<GraphCursorChunk>
|
|
|
|
|
/**
|
|
|
|
|
* @description Release an open cursor's pinned generation and server-side state.
|
|
|
|
|
* @param handle - From `graphCursorOpen`.
|
|
|
|
|
*/
|
|
|
|
|
graphCursorClose(handle: GraphCursorHandle): Promise<void>
|
|
|
|
|
|
|
|
|
|
/**
|
2026-06-22 09:54:01 -07:00
|
|
|
* @description Rank nodes by influence/importance over the VISIBLE graph (hidden
|
|
|
|
|
* tiers excluded by default, so the ranking reflects the public view). The
|
|
|
|
|
* ranking ALGORITHM is the provider's choice — the JS fallback uses PageRank; a
|
|
|
|
|
* native provider may use personalized PageRank, eigenvector centrality, etc. The
|
|
|
|
|
* intent ("which nodes matter most") is the contract, not the algorithm.
|
|
|
|
|
* @param options - Top-K + visibility (intent-level; no algorithm tuning).
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param generation - Optional as-of generation.
|
2026-06-22 09:54:01 -07:00
|
|
|
* @returns Per-node scores, descending.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
*/
|
2026-06-22 09:54:01 -07:00
|
|
|
rank(options: RankOptions, generation?: bigint): Promise<GraphScores>
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
/**
|
2026-06-22 09:54:01 -07:00
|
|
|
* @description Group related nodes into communities over the VISIBLE graph. The
|
|
|
|
|
* grouping ALGORITHM is the provider's choice — the JS fallback uses connected
|
|
|
|
|
* components; a native provider may use community detection (e.g. Louvain/Leiden).
|
|
|
|
|
* @param options - Directed-grouping toggle + visibility.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param generation - Optional as-of generation.
|
2026-06-22 09:54:01 -07:00
|
|
|
* @returns Per-node community labels + community count.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
*/
|
2026-06-22 09:54:01 -07:00
|
|
|
communities(options: CommunitiesOptions, generation?: bigint): Promise<GraphCommunities>
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
/**
|
2026-06-22 09:54:01 -07:00
|
|
|
* @description Find the best path between two nodes — fewest hops by default, or
|
|
|
|
|
* least summed edge weight with `by: 'weight'`. The pathfinding ALGORITHM is the
|
|
|
|
|
* provider's choice (JS fallback: BFS / Dijkstra).
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param fromInt - Start node int.
|
|
|
|
|
* @param toInt - End node int.
|
2026-06-22 09:54:01 -07:00
|
|
|
* @param options - Direction, cost basis (`by`), verb-type filter, max depth, visibility.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param generation - Optional as-of generation.
|
|
|
|
|
* @returns The path, or `null` if `toInt` is unreachable from `fromInt`.
|
|
|
|
|
*/
|
2026-06-22 09:54:01 -07:00
|
|
|
path(fromInt: bigint, toInt: bigint, options: PathOptions, generation?: bigint): Promise<GraphPath | null>
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
/**
|
2026-06-22 09:54:01 -07:00
|
|
|
* @description A bounded, representative SAMPLE of the neighborhood around the seeds
|
|
|
|
|
* (capped fan-out per node) — for dense-graph viz. Seeded for reproducibility.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param seeds - Start nodes (ints or an {@link OpaqueIdSet}).
|
|
|
|
|
* @param options - Depth, fan-out, RNG seed, visibility, node cap.
|
|
|
|
|
* @param generation - Optional as-of generation.
|
|
|
|
|
* @returns The sampled {@link Subgraph}.
|
|
|
|
|
*/
|
2026-06-22 09:54:01 -07:00
|
|
|
sample(seeds: bigint[] | OpaqueIdSet, options: SampleOptions, generation?: bigint): Promise<Subgraph>
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
/**
|
2026-06-22 09:54:01 -07:00
|
|
|
* @description The top-K most-connected nodes over the VISIBLE graph. The
|
|
|
|
|
* connectivity metric is the provider's choice (JS fallback: edge-count degree).
|
|
|
|
|
* @param options - Top-K, direction, visibility.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
* @param generation - Optional as-of generation.
|
2026-06-22 09:54:01 -07:00
|
|
|
* @returns Per-node connectivity scores, descending.
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
*/
|
2026-06-22 09:54:01 -07:00
|
|
|
mostConnected(options: MostConnectedOptions, generation?: bigint): Promise<GraphScores>
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
}
|
|
|
|
|
|
feat(8.0): generational MVCC storage + Datomic-style Db API (now/transact/asOf/with/persist)
One mechanism replaces the COW and versioning subsystems: immutable
generation-stamped records behind a Datomic-style database value.
Record layer (src/db/generationStore.ts):
- Monotonic generation counter in _system/generation.json; bumped once per
transact() commit and once per single-operation write (storage hook), so
brain.generation() is always a meaningful watermark.
- Commit protocol: stage before-images + tx.json delta -> fsync -> execute
batch via TransactionManager -> atomic tmp+rename of _system/manifest.json
(the rename IS the commit point) -> append _system/tx-log.jsonl.
- Crash recovery on open rolls uncommitted generations back byte-identically
and forces an index rebuild; refcounted pins gate compactHistory(), which
records a horizon (asOf below it throws GenerationCompactedError).
Db API:
- Db: get/find/search/related pinned at a generation, with() speculative
overlays, since() diffs, persist() hard-link-farm snapshots, timestamp,
generation, release() + FinalizationRegistry backstop.
- Brainy: now() O(1) pin, transact(ops, {meta, ifAtGeneration}) atomic batch
(GenerationConflictError CAS), asOf(generation|Date|path), restore(path,
{confirm}), compactHistory(), generation(), static open() + load().
- get()/metadata find()/related() are fully correct at any reachable pinned
generation; index-accelerated queries at historical generations throw
NotYetSupportedAtHistoricalGenerationError - never silently-wrong results.
- VersionedIndexProvider (generation/isGenerationVisible/pin/release) in
plugin.ts: feature-detected, balanced pin/release in lockstep with Db
lifecycle, post-commit applier + replay-gap model documented.
Storage primitives (BaseStorage + filesystem/memory adapters): raw-object
read/write/list/remove, fsync barrier, noun/verb raw before-image capture,
tx-log append, snapshotToDirectory (hard-link farm; byte-copy fallback and
append-in-place exceptions), restoreFromDirectory + derived-state reload.
Proof suite (tests/integration/db-mvcc.test.ts + tests/unit/db/): isolation
across 200 mutations, batch atomicity under injected execution failure,
ifAtGeneration CAS, snapshot immunity to source mutation, compaction safety
under pins, with() overlay isolation, generation monotonicity across
reopen, crash consistency through the real recovery path, and versioned
provider pin/release balance. Design record in
docs/ADR-001-generational-mvcc.md.
2026-06-10 14:14:07 -07:00
|
|
|
/**
|
|
|
|
|
* Optional capability interface for index providers that maintain versioned
|
|
|
|
|
* (generation-aware) internal state — the provider-side half of Brainy 8.0's
|
|
|
|
|
* generational MVCC contract. Implemented by native providers whose storage
|
|
|
|
|
* engines keep immutable versions (e.g. LSM snapshots); Brainy's own JS
|
|
|
|
|
* indexes do not implement it (they are rebuilt from the storage records,
|
|
|
|
|
* which carry the versioning).
|
|
|
|
|
*
|
|
|
|
|
* **Detection.** Brainy feature-detects this interface on every registered
|
|
|
|
|
* index provider (`'vector'`, `'metadataIndex'`, `'graphIndex'`): when a
|
|
|
|
|
* provider exposes `pin`/`release` functions, Brainy calls them in lockstep
|
|
|
|
|
* with `Db` lifecycle — `pin(g)` when a `Db` value pins generation `g`
|
|
|
|
|
* (`brain.now()`, `brain.transact()`, `brain.asOf()`), `release(g)` when that
|
|
|
|
|
* `Db` is released (explicitly or via the GC backstop). Pins are refcounted
|
|
|
|
|
* on the Brainy side; a provider may receive multiple `pin(g)` calls for the
|
|
|
|
|
* same generation and will receive exactly one matching `release(g)` per pin.
|
|
|
|
|
*
|
|
|
|
|
* **Consistency model (locked cross-team design).** Index providers are
|
|
|
|
|
* *post-commit appliers*: the storage-record commit (atomic manifest rename)
|
|
|
|
|
* is the source of truth, and provider index state is derived, applied after
|
|
|
|
|
* the commit point. On open, a provider compares its own persisted
|
|
|
|
|
* `generation()` against the store's committed generation and replays the
|
|
|
|
|
* gap from the storage records (or requests a rebuild) — there are no
|
|
|
|
|
* provider rollback hooks, because an uncommitted transaction is repaired at
|
|
|
|
|
* the storage layer before any index is opened. The explicit pin/release
|
|
|
|
|
* lifetime OVERRIDES any time-based snapshot retention the provider has
|
|
|
|
|
* (e.g. an LSM snapshot TTL): a pinned generation must stay readable until
|
|
|
|
|
* released, regardless of age.
|
|
|
|
|
*
|
|
|
|
|
* **Speculative reads.** `db.with(txData)` overlays are Brainy-side only —
|
|
|
|
|
* providers are never asked to read uncommitted/speculative state; they
|
|
|
|
|
* always serve committed generations.
|
|
|
|
|
*/
|
|
|
|
|
export interface VersionedIndexProvider {
|
|
|
|
|
/**
|
|
|
|
|
* @description The newest generation this provider's persisted index state
|
|
|
|
|
* reflects. Brainy compares it against the storage layer's committed
|
|
|
|
|
* generation on open to detect a replay gap (index behind storage after a
|
|
|
|
|
* crash between commit and index apply).
|
|
|
|
|
* @returns The provider's current generation as a `bigint` (u64 at the
|
|
|
|
|
* native boundary; Brainy's generation counter is a safe integer today).
|
|
|
|
|
*/
|
|
|
|
|
generation(): bigint
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description True ⇒ the provider can serve consistent reads at
|
|
|
|
|
* `generation` (segments retained). Combined with pin: pinning a VISIBLE
|
|
|
|
|
* generation guarantees it stays servable until release. Pinning an
|
|
|
|
|
* invisible generation is permitted (refcount-only) — Brainy serves that
|
|
|
|
|
* generation from canonical storage instead.
|
|
|
|
|
*
|
|
|
|
|
* Brainy consults this at pin time (the read-routing rule: a `Db` at
|
|
|
|
|
* generation `g` uses provider-accelerated reads when
|
|
|
|
|
* `isGenerationVisible(g)` was true at pin time; otherwise canonical
|
|
|
|
|
* generation records).
|
|
|
|
|
* @param generation - The generation a `Db` value is about to pin.
|
|
|
|
|
*/
|
|
|
|
|
isGenerationVisible(generation: bigint): boolean
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description Pin a generation: the provider must keep index state for
|
|
|
|
|
* `generation` readable until the matching {@link VersionedIndexProvider.release}
|
|
|
|
|
* call, overriding any time-based snapshot retention. Called once per
|
|
|
|
|
* Brainy-side pin (refcounted upstream — expect balanced pin/release pairs).
|
|
|
|
|
* @param generation - The generation a live `Db` value just pinned.
|
|
|
|
|
*/
|
|
|
|
|
pin(generation: bigint): void
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description Release one pin on `generation`. After the last release the
|
|
|
|
|
* provider may reclaim resources for that generation at its discretion.
|
|
|
|
|
* @param generation - The generation being released (matches a prior
|
|
|
|
|
* {@link VersionedIndexProvider.pin} call).
|
|
|
|
|
*/
|
|
|
|
|
release(generation: bigint): void
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description Feature-detection guard for {@link VersionedIndexProvider}.
|
|
|
|
|
* Brainy applies it to every registered index provider (vector, metadata,
|
|
|
|
|
* graph) when a `Db` value pins or releases a generation: providers exposing
|
|
|
|
|
* all four capability methods receive lockstep `pin`/`release` calls;
|
|
|
|
|
* everything else (including Brainy's own JS indexes) is skipped.
|
|
|
|
|
*
|
|
|
|
|
* @param candidate - A registered index provider instance.
|
|
|
|
|
* @returns Whether the candidate implements the versioned capability.
|
|
|
|
|
*/
|
|
|
|
|
export function isVersionedIndexProvider(
|
|
|
|
|
candidate: unknown
|
|
|
|
|
): candidate is VersionedIndexProvider {
|
|
|
|
|
if (candidate === null || typeof candidate !== 'object') {
|
|
|
|
|
return false
|
|
|
|
|
}
|
|
|
|
|
const c = candidate as Record<string, unknown>
|
|
|
|
|
return (
|
|
|
|
|
typeof c.generation === 'function' &&
|
|
|
|
|
typeof c.isGenerationVisible === 'function' &&
|
|
|
|
|
typeof c.pin === 'function' &&
|
|
|
|
|
typeof c.release === 'function'
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
feat(8.0): brain.graph.subgraph() + native-provider routing
Introduces the brain.graph namespace and its first op, subgraph() — bounded
multi-hop neighborhood extraction returning a hydrated GraphView (nodes with hop
depth + type/subtype, edges as Relations, a truncated flag). The one-call answer
to 'show me everything around this node, N hops out' the graph-viz path needs.
- brain.graph.subgraph(seeds, { depth, direction, type, subtype, includeInternal,
includeSystem, maxNodes, maxEdges, hydrateNodes }). seeds: id or id[].
- Feature-detect routing: when a GraphAccelerationProvider is registered
(isGraphAccelerationProvider on the 'graphAcceleration' provider) it routes to
native traverse() and hydrates the columnar Subgraph (node ints↔ids paired with
depth position-preserving, edge verb ints → Relations); otherwise a pure-TS BFS
fallback expands each frontier through the O(degree) related() adjacency.
- Both paths produce the identical GraphView, so CI exercises the shape via the
fallback; the native path is cross-layer-tested against cor's provider.
- New public types: GraphView, GraphNode, SubgraphOptions, GraphApi (the surface
grows: export/rank/path/communities follow). isGraphAccelerationProvider guard
added to plugin.ts.
Test: graph-subgraph.test.ts — depth tracking, direction, type filter, default-
hides-internal, node hydration on/off, maxNodes truncation, multi-seed.
2026-06-21 09:23:33 -07:00
|
|
|
/**
|
|
|
|
|
* @description Feature-detect an optional {@link GraphAccelerationProvider} (the
|
|
|
|
|
* native graph engine). Brainy probes the registered `'graphAcceleration'`
|
|
|
|
|
* provider with this: present ⇒ `brain.graph.*` routes to native `traverse` /
|
|
|
|
|
* cursor / analytics; absent ⇒ Brainy serves the same operations from its
|
|
|
|
|
* pure-TS adjacency fallback. Checks the two anchor methods (`traverse` +
|
|
|
|
|
* `graphCursorOpen`) — a partial implementation that lacks them is treated as
|
|
|
|
|
* absent (fall back rather than crash).
|
|
|
|
|
* @param candidate - The value registered under `'graphAcceleration'`, or undefined.
|
|
|
|
|
* @returns Whether `candidate` is a usable {@link GraphAccelerationProvider}.
|
|
|
|
|
*/
|
|
|
|
|
export function isGraphAccelerationProvider(
|
|
|
|
|
candidate: unknown
|
|
|
|
|
): candidate is GraphAccelerationProvider {
|
|
|
|
|
if (candidate === null || typeof candidate !== 'object') {
|
|
|
|
|
return false
|
|
|
|
|
}
|
|
|
|
|
const c = candidate as Record<string, unknown>
|
|
|
|
|
return typeof c.traverse === 'function' && typeof c.graphCursorOpen === 'function'
|
|
|
|
|
}
|
|
|
|
|
|
feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank
Completes brainy's #35 side. When the at-gen vector gate serves a filtered
semantic read, Brainy now ships the historically-correct candidate vectors so the
native provider reranks against them with zero per-vector FFI crossing — the
provider need not duplicate the per-generation retention Brainy already does.
- New `AtGenerationVectors { ids: BigInt64Array; vectors: Float32Array; dim }` on
the VectorIndexProvider.search options bag (row-major: vectors[i*dim..] == ids[i];
invariant vectors.length === ids.length*dim; ids = the candidate set; dim must
match the index dim). The built-in JS index ignores it.
- buildAtGenerationVectors resolves each universe id's vector AS OF the generation
(the record before-image, or live getNoun when untouched since the pin — not
readNounRaw, whose canonical path is empty under lazy-vector eviction), interns
the id via the shared mapper, and packs row-major. Absent/vectorless/wrong-dim
ids are dropped (not vector-rankable). Bounded by the filtered universe.
Shape + the four clarifications (row-major, ids-are-the-candidate-set, dim-asserted,
filtered-path-only) confirmed with cor in the handoff #35 thread. Inert until cor's
native at-gen rerank advertises isGenerationVisible (honesty guard).
Tested: the mock versioned provider now asserts it receives atGenerationVectors with
the row-major invariant, correct dim, and ids resolving back to the at-gen universe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 16:50:41 -07:00
|
|
|
/**
|
|
|
|
|
* Columnar at-`generation` candidate vectors for the 8.0 #35 FILTERED exact-rerank.
|
|
|
|
|
* Brainy resolves the per-generation vector before-images for the at-gen filtered
|
|
|
|
|
* universe (from its canonical generation records) and ships them flat, so a native
|
|
|
|
|
* provider reranks against the historically-correct vectors with ZERO per-vector FFI
|
|
|
|
|
* crossing — without the provider duplicating the per-gen retention Brainy already does.
|
|
|
|
|
*
|
|
|
|
|
* - **Row-major:** row `i` is `vectors[i*dim .. (i+1)*dim]` and belongs to `ids[i]`.
|
|
|
|
|
* INVARIANT: `vectors.length === ids.length * dim` (the provider asserts + refuses on mismatch).
|
|
|
|
|
* - **`ids` IS the candidate set:** entity ints (interned via the shared mapper) for the
|
|
|
|
|
* at-gen metadata∩graph filtered universe. The provider reranks EXACTLY these
|
|
|
|
|
* `(id, vector)` pairs and returns top-k by distance; any `allowedIds` passed alongside
|
|
|
|
|
* is redundant on this path (the provider may AND it as belt-and-suspenders).
|
|
|
|
|
* - **`dim`** must equal the provider's configured index dimension (asserted; mismatch → refuse).
|
|
|
|
|
*
|
|
|
|
|
* Supplied only when there is a BOUNDED filtered universe; the unfiltered-deep at-gen
|
|
|
|
|
* case stays the provider's own retained-segment ANN ({@link VersionedIndexProvider.isGenerationVisible}
|
|
|
|
|
* reports false there, so Brainy falls back to its materialization overlay).
|
|
|
|
|
*/
|
|
|
|
|
export interface AtGenerationVectors {
|
|
|
|
|
/** Entity ints (shared-mapper interned), one per candidate; the candidate set. */
|
|
|
|
|
ids: BigInt64Array
|
|
|
|
|
/** Flat row-major vectors: `vectors[i*dim .. (i+1)*dim]` is `ids[i]`'s at-gen vector. */
|
|
|
|
|
vectors: Float32Array
|
|
|
|
|
/** Vector dimension; MUST equal the provider's configured index dim. */
|
|
|
|
|
dim: number
|
|
|
|
|
}
|
|
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
/**
|
2026-06-09 12:58:26 -07:00
|
|
|
* The object returned by the `'vector'` provider factory — Brainy's vector
|
|
|
|
|
* index contract. Implementations include Brainy's own JS HNSW index and any
|
|
|
|
|
* native acceleration provider (e.g. cortex's Adaptive DiskANN).
|
|
|
|
|
*
|
|
|
|
|
* Brainy calls this surface via `this.index.*` plus the transactional add/remove
|
|
|
|
|
* operations. `enableCOW`, `getItem`, and `setPersistMode` are intentionally
|
|
|
|
|
* absent: Brainy guards each with feature-detection (`typeof x === 'function'`),
|
|
|
|
|
* so they are optional and not part of the required contract (Brainy's own JS
|
|
|
|
|
* HNSW index omits `setPersistMode`, for instance).
|
|
|
|
|
*
|
2026-06-10 11:29:05 -07:00
|
|
|
* **Provider key:** registered under `'vector'` — the only key Brainy
|
|
|
|
|
* consults for the vector index. The pre-8.0 `'hnsw'` and `'diskann'` keys
|
|
|
|
|
* are retired and never looked up.
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
*/
|
2026-06-09 12:58:26 -07:00
|
|
|
export interface VectorIndexProvider {
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
addItem(item: VectorDocument): Promise<string>
|
|
|
|
|
removeItem(id: string): Promise<boolean>
|
|
|
|
|
search(
|
|
|
|
|
queryVector: Vector,
|
|
|
|
|
k?: number,
|
|
|
|
|
filter?: (id: string) => Promise<boolean>,
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
options?: {
|
|
|
|
|
rerank?: { multiplier: number }
|
|
|
|
|
candidateIds?: string[]
|
|
|
|
|
/**
|
|
|
|
|
* Predicate-pushdown universe — restrict the result to this id-set, applied
|
|
|
|
|
* INSIDE the beam walk (walk traverses all nodes, collects only allowed), which
|
|
|
|
|
* recovers the filtered recall that post-filtering loses. A native provider with
|
|
|
|
|
* cor as the metadata index gets the `find()` universe as an {@link OpaqueIdSet}
|
|
|
|
|
* (zero id materialization); the pure-JS path gets a `ReadonlySet<string>`. Absent
|
|
|
|
|
* = no restriction. Optional — a provider that doesn't pushdown ignores it and
|
|
|
|
|
* falls back to `filter`.
|
|
|
|
|
*/
|
|
|
|
|
allowedIds?: OpaqueIdSet | ReadonlySet<string>
|
feat(8.0): asOf at-gen vector defer — provider-served historical semantic search (#35)
A filtered semantic read at a historical generation (db.asOf(g).find({ query, where }))
no longer unconditionally rebuilds an ephemeral JS-HNSW over every at-g vector
(O(n@G), the OOM ceiling the 2026-06-24 scaling audit flagged). When the vector
index is a VersionedIndexProvider that advertises isGenerationVisible(g), Brainy:
1. resolves the at-g metadata∩graph universe from the record-overlay path (no
materialization — it's the metadata-only historical find),
2. routes the vector leg to the provider with { allowedIds, generation }, and
3. composes the at-g entities ranked by the provider's at-g vector distance.
Without a versioned provider (the JS index, or a native one that refuses the
generation) it falls through to the existing materialization — unchanged.
- Seam (A): VectorIndexProvider.search gains an optional `generation?: bigint`
("omitted = now"), the vector mirror of the graph provider's trailing-gen + the
#46 allowedIds field. JsHnswVectorIndex accepts and ignores it (no per-gen
segments → refuse/fall-back posture; Brainy never routes a historical read there).
- Gate: Db.find tries the native at-gen vector path before materialize(); host
exposes canServeVectorAtGeneration + vectorSearchAtGeneration.
- generation is Brainy's u64 commit counter — the same value handed to the graph
index on writes (graphWriteGeneration), so it maps 1:1 to the native side.
Decided lockstep with the cor team (handoff #35 thread: (A) + vector-only). The
gen-g candidate-vector supply for cor's exact-rerank (part-3) is a separate,
non-blocking seam still being confirmed; the honesty guard keeps this path inactive
(falls through to materialize) until cor's native at-gen rerank is live.
Tested: provider routing + at-gen universe correctness (excludes future-born,
applies the filter) + page window via a mock versioned provider; seam-ignore on the
JS index; 38 db-mvcc/db-temporal historical-read tests green (materialize fallback).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 15:52:09 -07:00
|
|
|
/**
|
|
|
|
|
* As-of generation for historical (time-travel) reads — the vector-side
|
|
|
|
|
* mirror of the {@link GraphAccelerationProvider} trailing `generation?`.
|
|
|
|
|
* Omitted = "now" (the live index). When set, a {@link VersionedIndexProvider}
|
|
|
|
|
* serves the kNN / exact-rerank over its retained at-`generation` segments
|
|
|
|
|
* instead of the current vectors, so `db.asOf(g)` semantic queries need no
|
|
|
|
|
* O(n@G) JS-HNSW rebuild. Brainy only passes it when the provider advertised
|
|
|
|
|
* `isGenerationVisible(generation)` at pin time; a provider that does not
|
|
|
|
|
* honor it MUST refuse/fall back (never score current vectors as if at-gen) —
|
|
|
|
|
* Brainy then serves the historical leg from its own materialization. Brainy's
|
|
|
|
|
* `generation` is the same u64 counter handed to the graph index on writes.
|
|
|
|
|
*/
|
|
|
|
|
generation?: bigint
|
feat(8.0): #35 part-3 — supply at-gen candidate vectors for the native exact-rerank
Completes brainy's #35 side. When the at-gen vector gate serves a filtered
semantic read, Brainy now ships the historically-correct candidate vectors so the
native provider reranks against them with zero per-vector FFI crossing — the
provider need not duplicate the per-generation retention Brainy already does.
- New `AtGenerationVectors { ids: BigInt64Array; vectors: Float32Array; dim }` on
the VectorIndexProvider.search options bag (row-major: vectors[i*dim..] == ids[i];
invariant vectors.length === ids.length*dim; ids = the candidate set; dim must
match the index dim). The built-in JS index ignores it.
- buildAtGenerationVectors resolves each universe id's vector AS OF the generation
(the record before-image, or live getNoun when untouched since the pin — not
readNounRaw, whose canonical path is empty under lazy-vector eviction), interns
the id via the shared mapper, and packs row-major. Absent/vectorless/wrong-dim
ids are dropped (not vector-rankable). Bounded by the filtered universe.
Shape + the four clarifications (row-major, ids-are-the-candidate-set, dim-asserted,
filtered-path-only) confirmed with cor in the handoff #35 thread. Inert until cor's
native at-gen rerank advertises isGenerationVisible (honesty guard).
Tested: the mock versioned provider now asserts it receives atGenerationVectors with
the row-major invariant, correct dim, and ids resolving back to the at-gen universe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 16:50:41 -07:00
|
|
|
/**
|
|
|
|
|
* 8.0 #35 part-3: the at-`generation` candidate vectors for the FILTERED
|
|
|
|
|
* exact-rerank — Brainy supplies the historically-correct vectors so the
|
|
|
|
|
* provider need not retain them. Present only alongside `generation` on the
|
|
|
|
|
* filtered at-gen path; the provider reranks `atGenerationVectors.ids` by
|
|
|
|
|
* distance and returns top-k. See {@link AtGenerationVectors}. The built-in
|
|
|
|
|
* JS index ignores it (it serves "now" only).
|
|
|
|
|
*/
|
|
|
|
|
atGenerationVectors?: AtGenerationVectors
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
}
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
): Promise<Array<[string, number]>>
|
|
|
|
|
size(): number
|
|
|
|
|
clear(): void
|
|
|
|
|
rebuild(options?: any): Promise<void>
|
|
|
|
|
flush(): Promise<number>
|
|
|
|
|
getPersistMode(): 'immediate' | 'deferred'
|
2026-06-30 13:37:23 -07:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL. A native provider returns true from the moment its
|
|
|
|
|
* `init()` detects a large epoch-drift until its background
|
|
|
|
|
* build-new→verify→swap has verified-and-swapped. While true, brainy SKIPS its
|
|
|
|
|
* own rebuild for this provider and lets the provider's non-blocking background
|
|
|
|
|
* migration own the index (the no-freeze path); the provider serves correct
|
|
|
|
|
* reads from canonical meanwhile. Mirrors `isReady?()` / `init?()` on the
|
|
|
|
|
* other index providers.
|
|
|
|
|
*/
|
|
|
|
|
isMigrating?(): boolean
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
}
|
|
|
|
|
|
2026-05-28 14:29:12 -07:00
|
|
|
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
/**
|
|
|
|
|
* The `'entityIdMapper'` provider — a drop-in for `EntityIdMapper`. Injected
|
|
|
|
|
* into the TypeScript `MetadataIndexManager` when a native metadata index is
|
|
|
|
|
* not also registered; that coordinator calls this full surface (incl.
|
|
|
|
|
* `getAllIntIds`, the all-ids universe for negation / `exists:false` filters).
|
|
|
|
|
*/
|
|
|
|
|
export interface EntityIdMapperProvider {
|
|
|
|
|
init(): Promise<void>
|
2026-06-23 12:02:17 -07:00
|
|
|
/**
|
|
|
|
|
* @description OPTIONAL: reload the uuid↔int mapping from the CURRENT storage,
|
|
|
|
|
* discarding in-memory state — called by `brain.restore()` AFTER the storage
|
|
|
|
|
* has been replaced from a snapshot and BEFORE the graph index rebuilds, so
|
|
|
|
|
* the graph resolves each verb endpoint through a mapper that reflects the
|
|
|
|
|
* snapshot's assignments (without it, native adjacency edges — keyed on these
|
|
|
|
|
* ints — resolve to stale/missing ints and are silently dropped). A native
|
|
|
|
|
* mapper reloads from its restored binary KV; the JS fallback re-reads its
|
|
|
|
|
* persisted metadata. Optional so a mapper that already reloads via `init()`
|
|
|
|
|
* stays compatible — `restore()` falls back to `init()` when this is absent.
|
|
|
|
|
*/
|
|
|
|
|
rebuild?(): Promise<void>
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
getOrAssign(uuid: string): number
|
|
|
|
|
getUuid(intId: number): string | undefined
|
|
|
|
|
getInt(uuid: string): number | undefined
|
|
|
|
|
remove(uuid: string): boolean
|
|
|
|
|
flush(): Promise<void>
|
|
|
|
|
clear(): Promise<void>
|
|
|
|
|
getAllIntIds(): number[]
|
|
|
|
|
intsIterableToUuids(ints: Iterable<number>): string[]
|
feat(8.0): GraphAccelerationProvider contract — the native graph-engine seam
Defines the optional native graph-acceleration provider (cor 3.0) that
brainy feature-detects and routes brain.graph.* / related({node}) /
find({connected}) to, falling back to pure-TS adjacency when absent. This is
the published seam cor wires its native build against (mirrors the existing
VersionedIndexProvider pattern: defined in plugin.ts, exported from index.ts,
implemented externally, feature-detected at runtime).
Contract (locked cross-team in the design thread):
- GraphAccelerationProvider: traverse, edgesForNode, graphCursorOpen/Next/Close,
pageRank, connectedComponents, shortestPath, neighborhoodSample, topByDegree.
Every read op is generation-aware (optional trailing generation?: bigint) so
db.asOf(g).graph.* resolves historically; omitted = now.
- Subgraph: columnar wire format (BigInt64Array node/edge columns, Uint16Array
edgeTypes, optional Uint8Array nodeDepth, truncated flag) — parallel typed
arrays, never array-of-objects; brainy maps u64<->UUID lazily for rendered rows.
- OpaqueIdSet: a find() universe forwarded opaquely into traverse/search for the
zero-crossing query->expand fusion (brainy never inspects it; the provider
version-tags + validates the envelope).
- VectorIndexProvider.search gains allowedIds?: OpaqueIdSet | ReadonlySet<string>
(predicate pushdown — recovers filtered recall lost to post-filtering).
- EntityIdMapperProvider gains entityIntsToUuids(BigInt64Array) — the bigint batch
reverse resolver for Subgraph.nodes; implemented on the JS mapper (TS fallback).
All new public types exported from index.ts. Test: entityIntsToUuids round-trip
+ order-preservation + empty-int sentinel.
2026-06-21 08:12:22 -07:00
|
|
|
/**
|
|
|
|
|
* @description Batch reverse-resolve u64 entity ints → UUID strings — the
|
|
|
|
|
* bigint counterpart of {@link EntityIdMapperProvider.intsIterableToUuids},
|
|
|
|
|
* for the native graph engine ({@link GraphAccelerationProvider}, whose
|
|
|
|
|
* `Subgraph.nodes` is a `BigInt64Array`). Brainy resolves lazily — only the
|
|
|
|
|
* rows a caller renders — but viz/export render many at once, so one batch
|
|
|
|
|
* crossing beats N. Order-preserving (one entry per input). A native mapper
|
|
|
|
|
* resolves every int from its binary int↔uuid store; the JS fallback resolves
|
|
|
|
|
* assigned ints and yields `''` for a never-assigned int (which does not occur
|
|
|
|
|
* for graph-engine results, since those only reference assigned ints).
|
|
|
|
|
* @param nodeInts - Entity ints as returned in a {@link Subgraph}.
|
|
|
|
|
* @returns One UUID per input, in order.
|
|
|
|
|
*/
|
|
|
|
|
entityIntsToUuids(nodeInts: BigInt64Array): string[]
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
readonly size: number
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The `'cache'` provider — a drop-in for `UnifiedCache`. Brainy installs it as
|
|
|
|
|
* the global cache (`setGlobalCache`) and calls this surface via
|
|
|
|
|
* `getGlobalCache()`.
|
|
|
|
|
*/
|
|
|
|
|
export interface CacheProvider {
|
|
|
|
|
getSync(key: string): any | undefined
|
refactor(8.0): final cleanup — drop HnswProvider alias + config.hnsw + 'hnsw' surface (scaffold step 6)
Step 6 of the brainy 8.0 rename scaffolding. Removes every legacy 'hnsw'-
flavoured public name introduced as a compat shim in steps 1-5. The
algorithm-neutral surface is now the only surface.
REMOVED — PHASE A (type aliases)
src/plugin.ts
- Removed `export type HnswProvider = VectorIndexProvider` alias.
- Removed `export type DiskAnnProvider = VectorIndexProvider` alias.
src/index.ts
- Removed `export const HNSWIndex = JsHnswVectorIndex` alias.
src/types/brainy.types.ts
- Removed `BrainyStats.indexHealth.hnsw` field (the new `vector` field
carries the same boolean).
src/brainy.ts
- `stats()` no longer emits the legacy `hnsw` field on `indexHealth`.
REMOVED — PHASE B (storage adapter rename, 8 adapters)
Repo-wide rename: `saveHNSWData` → `saveVectorIndexData` and
`getHNSWData` → `getVectorIndexData` across:
- src/storage/adapters/baseStorageAdapter.ts (abstract declarations)
- src/storage/adapters/fileSystemStorage.ts
- src/storage/adapters/gcsStorage.ts
- src/storage/adapters/r2Storage.ts
- src/storage/adapters/s3CompatibleStorage.ts
- src/storage/adapters/azureBlobStorage.ts
- src/storage/adapters/opfsStorage.ts
- src/storage/adapters/memoryStorage.ts
- src/storage/adapters/historicalStorageAdapter.ts
- src/brainy.ts (call sites)
- src/hnsw/hnswIndex.ts + src/hnsw/typeAwareHNSWIndex.ts (call sites)
The default-delegation wrappers added in scaffold step 4 are removed
(would have been duplicate declarations after the rename).
REMOVED — PHASE C (cache category 'hnsw')
src/utils/unifiedCache.ts
- Cache-category union narrowed from
'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
to
'vectors' | 'metadata' | 'embedding' | 'other'
- typeAccessCounts, typeSizes, typeCounts, accessRatios, sizeRatios,
and the fairness-check iterator all drop the 'hnsw' key.
- Per planning § 2.5: pre-8.0 'hnsw' cache entries are an in-memory
category. Cache is rebuildable; entries naturally don't exist after
a restart, so no migration path is needed.
src/hnsw/hnswIndex.ts
- 3 cache.set() call sites migrated from category 'hnsw' to 'vectors'.
- Cache key prefix `hnsw:vector:` → `vector:`.
- typeCounts/typeSizes/typeAccessCounts accessor renames
`.hnsw` → `.vectors`.
tests/unit/utils/unifiedCache-eviction.test.ts
- Test fixtures updated: cache.set(..., 'hnsw', ...) → cache.set(..., 'vectors', ...).
- Stats assertion `stats.typeSizes.hnsw` → `stats.typeSizes.vectors`.
REMOVED — PHASE D (config.hnsw)
src/types/brainy.types.ts
- Removed `BrainyConfig.hnsw` field entirely. The 8.0 surface is
`BrainyConfig.vector.{recall, quantization, vectorStorage, advanced}`.
src/brainy.ts
- normalizeConfig() no longer emits a `hnsw` field on Required<BrainyConfig>.
- setupIndex() rewired:
- Reads from `this.config.vector` (not `this.config.hnsw`).
- Calls `resolveJsHnswConfig(this.config.vector)` to translate the
`recall` preset into M / efConstruction / efSearch knobs (with
`advanced.hnsw` overrides winning when supplied).
- Imports `resolveJsHnswConfig` from './utils/recallPreset.js'.
NOT IN THIS COMMIT (deliberately)
- Persisted file path migration `_system/hnsw-*.json` →
`_system/vector-index-*.json`. Requires dual-read logic on boot
across 8 storage adapters. Per integration doc lines 531-539:
"Reader accepts either spelling on load; writer emits the new spelling
only; brains self-migrate on the next persist after upgrade." This is
a separate body of work and lands in a follow-up commit before 8.0 GA.
- `config.hnswPersistMode` top-level field is unchanged. It's not in
the rename inventory; a future commit can fold it into
`config.vector.persistMode` if desired.
- strictConfig enforcement wiring. The field is accepted by
normalizeConfig() with default 'warn'; the actual warning emission at
knob-mismatch sites lands when the config-resolution layer is touched
for the persisted-path migration.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit (one test fixture updated to use the new
category name; assertion still passes after fixture rename)
- npm run build: clean
The 8.0 PR's user-facing surface is now algorithm-neutral end-to-end:
VectorIndexProvider, config.vector.recall, JsHnswVectorIndex,
saveVectorIndexData, 'vectors' cache category, indexHealth.vector,
strictConfig — all standalone. No legacy 'hnsw' name remains in any
public type or method signature.
2026-06-09 13:28:53 -07:00
|
|
|
set(key: string, data: any, type: 'vectors' | 'metadata' | 'embedding' | 'other', size: number, rebuildCost?: number): void
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
delete(key: string): boolean
|
|
|
|
|
deleteByPrefix(prefix: string): number
|
refactor(8.0): final cleanup — drop HnswProvider alias + config.hnsw + 'hnsw' surface (scaffold step 6)
Step 6 of the brainy 8.0 rename scaffolding. Removes every legacy 'hnsw'-
flavoured public name introduced as a compat shim in steps 1-5. The
algorithm-neutral surface is now the only surface.
REMOVED — PHASE A (type aliases)
src/plugin.ts
- Removed `export type HnswProvider = VectorIndexProvider` alias.
- Removed `export type DiskAnnProvider = VectorIndexProvider` alias.
src/index.ts
- Removed `export const HNSWIndex = JsHnswVectorIndex` alias.
src/types/brainy.types.ts
- Removed `BrainyStats.indexHealth.hnsw` field (the new `vector` field
carries the same boolean).
src/brainy.ts
- `stats()` no longer emits the legacy `hnsw` field on `indexHealth`.
REMOVED — PHASE B (storage adapter rename, 8 adapters)
Repo-wide rename: `saveHNSWData` → `saveVectorIndexData` and
`getHNSWData` → `getVectorIndexData` across:
- src/storage/adapters/baseStorageAdapter.ts (abstract declarations)
- src/storage/adapters/fileSystemStorage.ts
- src/storage/adapters/gcsStorage.ts
- src/storage/adapters/r2Storage.ts
- src/storage/adapters/s3CompatibleStorage.ts
- src/storage/adapters/azureBlobStorage.ts
- src/storage/adapters/opfsStorage.ts
- src/storage/adapters/memoryStorage.ts
- src/storage/adapters/historicalStorageAdapter.ts
- src/brainy.ts (call sites)
- src/hnsw/hnswIndex.ts + src/hnsw/typeAwareHNSWIndex.ts (call sites)
The default-delegation wrappers added in scaffold step 4 are removed
(would have been duplicate declarations after the rename).
REMOVED — PHASE C (cache category 'hnsw')
src/utils/unifiedCache.ts
- Cache-category union narrowed from
'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
to
'vectors' | 'metadata' | 'embedding' | 'other'
- typeAccessCounts, typeSizes, typeCounts, accessRatios, sizeRatios,
and the fairness-check iterator all drop the 'hnsw' key.
- Per planning § 2.5: pre-8.0 'hnsw' cache entries are an in-memory
category. Cache is rebuildable; entries naturally don't exist after
a restart, so no migration path is needed.
src/hnsw/hnswIndex.ts
- 3 cache.set() call sites migrated from category 'hnsw' to 'vectors'.
- Cache key prefix `hnsw:vector:` → `vector:`.
- typeCounts/typeSizes/typeAccessCounts accessor renames
`.hnsw` → `.vectors`.
tests/unit/utils/unifiedCache-eviction.test.ts
- Test fixtures updated: cache.set(..., 'hnsw', ...) → cache.set(..., 'vectors', ...).
- Stats assertion `stats.typeSizes.hnsw` → `stats.typeSizes.vectors`.
REMOVED — PHASE D (config.hnsw)
src/types/brainy.types.ts
- Removed `BrainyConfig.hnsw` field entirely. The 8.0 surface is
`BrainyConfig.vector.{recall, quantization, vectorStorage, advanced}`.
src/brainy.ts
- normalizeConfig() no longer emits a `hnsw` field on Required<BrainyConfig>.
- setupIndex() rewired:
- Reads from `this.config.vector` (not `this.config.hnsw`).
- Calls `resolveJsHnswConfig(this.config.vector)` to translate the
`recall` preset into M / efConstruction / efSearch knobs (with
`advanced.hnsw` overrides winning when supplied).
- Imports `resolveJsHnswConfig` from './utils/recallPreset.js'.
NOT IN THIS COMMIT (deliberately)
- Persisted file path migration `_system/hnsw-*.json` →
`_system/vector-index-*.json`. Requires dual-read logic on boot
across 8 storage adapters. Per integration doc lines 531-539:
"Reader accepts either spelling on load; writer emits the new spelling
only; brains self-migrate on the next persist after upgrade." This is
a separate body of work and lands in a follow-up commit before 8.0 GA.
- `config.hnswPersistMode` top-level field is unchanged. It's not in
the rename inventory; a future commit can fold it into
`config.vector.persistMode` if desired.
- strictConfig enforcement wiring. The field is accepted by
normalizeConfig() with default 'warn'; the actual warning emission at
knob-mismatch sites lands when the config-resolution layer is touched
for the persisted-path migration.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit (one test fixture updated to use the new
category name; assertion still passes after fixture rename)
- npm run build: clean
The 8.0 PR's user-facing surface is now algorithm-neutral end-to-end:
VectorIndexProvider, config.vector.recall, JsHnswVectorIndex,
saveVectorIndexData, 'vectors' cache category, indexHealth.vector,
strictConfig — all standalone. No legacy 'hnsw' name remains in any
public type or method signature.
2026-06-09 13:28:53 -07:00
|
|
|
clear(type?: 'vectors' | 'metadata' | 'embedding' | 'other'): void
|
feat: export provider contracts for the plugin surface brainy consumes
Native accelerators (cortex) register providers for metadataIndex, graphIndex,
hnsw, entityIdMapper, cache, columnStore, and aggregation. Until now the exact
method/property surface brainy calls on each was implicit — a provider could
drop a member brainy depends on and only fail at runtime when that path ran.
This defines and exports the provider contracts from the stable
@soulcraft/brainy/plugin entrypoint:
- MetadataIndexProvider, GraphIndexProvider, HnswProvider,
EntityIdMapperProvider, CacheProvider — each typed as exactly the surface
brainy calls (optional/feature-detected members like HNSW setPersistMode and
enableCOW are intentionally excluded).
- Re-exports ColumnStoreProvider and AggregationProvider (and the aggregate
types) from the same entrypoint so a plugin author can import the whole
provider surface from one place.
Brainy's own baseline classes now `implements` these contracts
(MetadataIndexManager, GraphAdjacencyIndex, HNSWIndex, EntityIdMapper,
UnifiedCache), so the interfaces can never silently diverge from what brainy
ships — and any provider that declares `implements` gets a compile error the
moment brainy starts requiring a new member.
The embeddings/embedBatch providers stay typed by the existing EmbeddingFunction
(no duplicate interface added). Type-only changes; no runtime behavior change.
2026-05-27 14:45:40 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// The `'embeddings'` / `'embedBatch'` providers are function-shaped and already
|
|
|
|
|
// typed by the existing `EmbeddingFunction` (see `coreTypes.ts`), which Brainy
|
|
|
|
|
// uses at the `getProvider('embeddings')` call site. No separate interface is
|
|
|
|
|
// added here to avoid a duplicate, unwired contract.
|
|
|
|
|
|
feat: graph link compression — delta-varint connections (2.4.0 #3)
Cortex registers a graph:compression provider exposing `encode`/`decode`
for HNSW connection lists (delta-varint, ~4× smaller edges than the
JSON-UUID-array shape brainy has persisted since the beginning). This
wires the consumer side without changing the saveHNSWData/getHNSWData
adapter signatures.
Architecture:
- NEW ConnectionsCodec (src/hnsw/connectionsCodec.ts) — translates between
UUID-keyed Map<level, Set<UUID>> and a single compact per-node buffer
via the provider + stable EntityIdMapper. Custom wire format:
[count: u8] [[level: u8] [len: u32 LE] [bytes...]]*. One blob per node
regardless of level count, so the storage I/O is identical to the
legacy path (one saveHNSWData call) plus a single saveBinaryBlob.
- HNSWIndex changes — a `connectionsCodec: ConnectionsCodec | null` field
with `setConnectionsCodec()` setter. THREE save sites (deferred flush,
immediate-mode entity persist, immediate-mode neighbor updates) now go
through a single `persistNodeConnections(nodeId, noun)` helper: when the
codec is wired AND the storage adapter exposes saveBinaryBlob, it
encodes the connections, stores them at `_hnsw_conn/<nodeId>`, and
records `connections: {}` in saveHNSWData as the marker. Otherwise the
legacy JSON-array path is taken. TWO load sites use a matching
`restoreNodeConnections` helper that tries loadBinaryBlob first and
falls back to the legacy hnswData.connections field on miss / decode
error. Format convergence is lazy: pre-2.4.0 nodes still load via the
legacy path, then write the compressed form on next dirty save.
- brainy.ts wireConnectionsCodec — runs alongside wireMmapVectorBackend
during init. Activates when (a) the graph:compression provider is
registered and (b) the metadata index exposes its idMapper. Unlike the
mmap-vector backend, this layer engages on EVERY brainy 7.25.0
adapter — the blob primitive itself is universal; only the codec
presence gates activation.
- Provider interface GraphCompressionProvider in plugin.ts — encode +
decode static signatures. Brainy depends on the interface; cortex's
registered { encode: encodeConnections, decode: decodeConnections }
satisfies it structurally.
Tests (1437 total, +4 vs prior tip):
- tests/unit/hnsw/connections-codec.test.ts — 4 unit tests with a JSON
mock provider: single-level round-trip, empty-Map round-trip (one-byte
buffer), multi-level fan-out round-trip, and silent-drop of UUIDs the
decoding idMapper no longer knows. The real cross-language byte
format is exercised when cortex 2.4.0 wires its delta-varint
encode/decode in.
This completes the brainy half of the 2.4.0 storage foundation: stable
ids (#23), mmap vectors (#24), graph link compression (#25), and
column-store interchange (#26). Coordinated release as brainy 7.26.0 +
cortex 2.4.0 follows.
2026-05-28 10:37:01 -07:00
|
|
|
/**
|
|
|
|
|
* The `'graph:compression'` provider — pure-function encode/decode for HNSW
|
|
|
|
|
* connection lists as compact delta-varint byte sequences (cortex's
|
|
|
|
|
* `encodeConnections` / `decodeConnections`).
|
|
|
|
|
*
|
2026-06-09 13:07:56 -07:00
|
|
|
* Brainy's `JsHnswVectorIndex` consumes this via a `ConnectionsCodec` that translates
|
feat: graph link compression — delta-varint connections (2.4.0 #3)
Cortex registers a graph:compression provider exposing `encode`/`decode`
for HNSW connection lists (delta-varint, ~4× smaller edges than the
JSON-UUID-array shape brainy has persisted since the beginning). This
wires the consumer side without changing the saveHNSWData/getHNSWData
adapter signatures.
Architecture:
- NEW ConnectionsCodec (src/hnsw/connectionsCodec.ts) — translates between
UUID-keyed Map<level, Set<UUID>> and a single compact per-node buffer
via the provider + stable EntityIdMapper. Custom wire format:
[count: u8] [[level: u8] [len: u32 LE] [bytes...]]*. One blob per node
regardless of level count, so the storage I/O is identical to the
legacy path (one saveHNSWData call) plus a single saveBinaryBlob.
- HNSWIndex changes — a `connectionsCodec: ConnectionsCodec | null` field
with `setConnectionsCodec()` setter. THREE save sites (deferred flush,
immediate-mode entity persist, immediate-mode neighbor updates) now go
through a single `persistNodeConnections(nodeId, noun)` helper: when the
codec is wired AND the storage adapter exposes saveBinaryBlob, it
encodes the connections, stores them at `_hnsw_conn/<nodeId>`, and
records `connections: {}` in saveHNSWData as the marker. Otherwise the
legacy JSON-array path is taken. TWO load sites use a matching
`restoreNodeConnections` helper that tries loadBinaryBlob first and
falls back to the legacy hnswData.connections field on miss / decode
error. Format convergence is lazy: pre-2.4.0 nodes still load via the
legacy path, then write the compressed form on next dirty save.
- brainy.ts wireConnectionsCodec — runs alongside wireMmapVectorBackend
during init. Activates when (a) the graph:compression provider is
registered and (b) the metadata index exposes its idMapper. Unlike the
mmap-vector backend, this layer engages on EVERY brainy 7.25.0
adapter — the blob primitive itself is universal; only the codec
presence gates activation.
- Provider interface GraphCompressionProvider in plugin.ts — encode +
decode static signatures. Brainy depends on the interface; cortex's
registered { encode: encodeConnections, decode: decodeConnections }
satisfies it structurally.
Tests (1437 total, +4 vs prior tip):
- tests/unit/hnsw/connections-codec.test.ts — 4 unit tests with a JSON
mock provider: single-level round-trip, empty-Map round-trip (one-byte
buffer), multi-level fan-out round-trip, and silent-drop of UUIDs the
decoding idMapper no longer knows. The real cross-language byte
format is exercised when cortex 2.4.0 wires its delta-varint
encode/decode in.
This completes the brainy half of the 2.4.0 storage foundation: stable
ids (#23), mmap vectors (#24), graph link compression (#25), and
column-store interchange (#26). Coordinated release as brainy 7.26.0 +
cortex 2.4.0 follows.
2026-05-28 10:37:01 -07:00
|
|
|
* UUIDs to stable int slots via the `EntityIdMapper`, encodes, and persists
|
|
|
|
|
* the compressed bytes through the binary-blob primitive. On load, the blob
|
|
|
|
|
* is fetched + decoded back into UUID sets — `setConnectionsCodec()` on
|
2026-06-09 13:07:56 -07:00
|
|
|
* `JsHnswVectorIndex` is the injection point. Read path is dual-format: when no blob
|
feat: graph link compression — delta-varint connections (2.4.0 #3)
Cortex registers a graph:compression provider exposing `encode`/`decode`
for HNSW connection lists (delta-varint, ~4× smaller edges than the
JSON-UUID-array shape brainy has persisted since the beginning). This
wires the consumer side without changing the saveHNSWData/getHNSWData
adapter signatures.
Architecture:
- NEW ConnectionsCodec (src/hnsw/connectionsCodec.ts) — translates between
UUID-keyed Map<level, Set<UUID>> and a single compact per-node buffer
via the provider + stable EntityIdMapper. Custom wire format:
[count: u8] [[level: u8] [len: u32 LE] [bytes...]]*. One blob per node
regardless of level count, so the storage I/O is identical to the
legacy path (one saveHNSWData call) plus a single saveBinaryBlob.
- HNSWIndex changes — a `connectionsCodec: ConnectionsCodec | null` field
with `setConnectionsCodec()` setter. THREE save sites (deferred flush,
immediate-mode entity persist, immediate-mode neighbor updates) now go
through a single `persistNodeConnections(nodeId, noun)` helper: when the
codec is wired AND the storage adapter exposes saveBinaryBlob, it
encodes the connections, stores them at `_hnsw_conn/<nodeId>`, and
records `connections: {}` in saveHNSWData as the marker. Otherwise the
legacy JSON-array path is taken. TWO load sites use a matching
`restoreNodeConnections` helper that tries loadBinaryBlob first and
falls back to the legacy hnswData.connections field on miss / decode
error. Format convergence is lazy: pre-2.4.0 nodes still load via the
legacy path, then write the compressed form on next dirty save.
- brainy.ts wireConnectionsCodec — runs alongside wireMmapVectorBackend
during init. Activates when (a) the graph:compression provider is
registered and (b) the metadata index exposes its idMapper. Unlike the
mmap-vector backend, this layer engages on EVERY brainy 7.25.0
adapter — the blob primitive itself is universal; only the codec
presence gates activation.
- Provider interface GraphCompressionProvider in plugin.ts — encode +
decode static signatures. Brainy depends on the interface; cortex's
registered { encode: encodeConnections, decode: decodeConnections }
satisfies it structurally.
Tests (1437 total, +4 vs prior tip):
- tests/unit/hnsw/connections-codec.test.ts — 4 unit tests with a JSON
mock provider: single-level round-trip, empty-Map round-trip (one-byte
buffer), multi-level fan-out round-trip, and silent-drop of UUIDs the
decoding idMapper no longer knows. The real cross-language byte
format is exercised when cortex 2.4.0 wires its delta-varint
encode/decode in.
This completes the brainy half of the 2.4.0 storage foundation: stable
ids (#23), mmap vectors (#24), graph link compression (#25), and
column-store interchange (#26). Coordinated release as brainy 7.26.0 +
cortex 2.4.0 follows.
2026-05-28 10:37:01 -07:00
|
|
|
* exists for a node, the connections fall back to the legacy JSON-array path
|
refactor(8.0): final cleanup — drop HnswProvider alias + config.hnsw + 'hnsw' surface (scaffold step 6)
Step 6 of the brainy 8.0 rename scaffolding. Removes every legacy 'hnsw'-
flavoured public name introduced as a compat shim in steps 1-5. The
algorithm-neutral surface is now the only surface.
REMOVED — PHASE A (type aliases)
src/plugin.ts
- Removed `export type HnswProvider = VectorIndexProvider` alias.
- Removed `export type DiskAnnProvider = VectorIndexProvider` alias.
src/index.ts
- Removed `export const HNSWIndex = JsHnswVectorIndex` alias.
src/types/brainy.types.ts
- Removed `BrainyStats.indexHealth.hnsw` field (the new `vector` field
carries the same boolean).
src/brainy.ts
- `stats()` no longer emits the legacy `hnsw` field on `indexHealth`.
REMOVED — PHASE B (storage adapter rename, 8 adapters)
Repo-wide rename: `saveHNSWData` → `saveVectorIndexData` and
`getHNSWData` → `getVectorIndexData` across:
- src/storage/adapters/baseStorageAdapter.ts (abstract declarations)
- src/storage/adapters/fileSystemStorage.ts
- src/storage/adapters/gcsStorage.ts
- src/storage/adapters/r2Storage.ts
- src/storage/adapters/s3CompatibleStorage.ts
- src/storage/adapters/azureBlobStorage.ts
- src/storage/adapters/opfsStorage.ts
- src/storage/adapters/memoryStorage.ts
- src/storage/adapters/historicalStorageAdapter.ts
- src/brainy.ts (call sites)
- src/hnsw/hnswIndex.ts + src/hnsw/typeAwareHNSWIndex.ts (call sites)
The default-delegation wrappers added in scaffold step 4 are removed
(would have been duplicate declarations after the rename).
REMOVED — PHASE C (cache category 'hnsw')
src/utils/unifiedCache.ts
- Cache-category union narrowed from
'hnsw' | 'vectors' | 'metadata' | 'embedding' | 'other'
to
'vectors' | 'metadata' | 'embedding' | 'other'
- typeAccessCounts, typeSizes, typeCounts, accessRatios, sizeRatios,
and the fairness-check iterator all drop the 'hnsw' key.
- Per planning § 2.5: pre-8.0 'hnsw' cache entries are an in-memory
category. Cache is rebuildable; entries naturally don't exist after
a restart, so no migration path is needed.
src/hnsw/hnswIndex.ts
- 3 cache.set() call sites migrated from category 'hnsw' to 'vectors'.
- Cache key prefix `hnsw:vector:` → `vector:`.
- typeCounts/typeSizes/typeAccessCounts accessor renames
`.hnsw` → `.vectors`.
tests/unit/utils/unifiedCache-eviction.test.ts
- Test fixtures updated: cache.set(..., 'hnsw', ...) → cache.set(..., 'vectors', ...).
- Stats assertion `stats.typeSizes.hnsw` → `stats.typeSizes.vectors`.
REMOVED — PHASE D (config.hnsw)
src/types/brainy.types.ts
- Removed `BrainyConfig.hnsw` field entirely. The 8.0 surface is
`BrainyConfig.vector.{recall, quantization, vectorStorage, advanced}`.
src/brainy.ts
- normalizeConfig() no longer emits a `hnsw` field on Required<BrainyConfig>.
- setupIndex() rewired:
- Reads from `this.config.vector` (not `this.config.hnsw`).
- Calls `resolveJsHnswConfig(this.config.vector)` to translate the
`recall` preset into M / efConstruction / efSearch knobs (with
`advanced.hnsw` overrides winning when supplied).
- Imports `resolveJsHnswConfig` from './utils/recallPreset.js'.
NOT IN THIS COMMIT (deliberately)
- Persisted file path migration `_system/hnsw-*.json` →
`_system/vector-index-*.json`. Requires dual-read logic on boot
across 8 storage adapters. Per integration doc lines 531-539:
"Reader accepts either spelling on load; writer emits the new spelling
only; brains self-migrate on the next persist after upgrade." This is
a separate body of work and lands in a follow-up commit before 8.0 GA.
- `config.hnswPersistMode` top-level field is unchanged. It's not in
the rename inventory; a future commit can fold it into
`config.vector.persistMode` if desired.
- strictConfig enforcement wiring. The field is accepted by
normalizeConfig() with default 'warn'; the actual warning emission at
knob-mismatch sites lands when the config-resolution layer is touched
for the persisted-path migration.
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit (one test fixture updated to use the new
category name; assertion still passes after fixture rename)
- npm run build: clean
The 8.0 PR's user-facing surface is now algorithm-neutral end-to-end:
VectorIndexProvider, config.vector.recall, JsHnswVectorIndex,
saveVectorIndexData, 'vectors' cache category, indexHealth.vector,
strictConfig — all standalone. No legacy 'hnsw' name remains in any
public type or method signature.
2026-06-09 13:28:53 -07:00
|
|
|
* embedded in `saveVectorIndexData`, so pre-2.4.0 indexes keep loading unchanged
|
feat: graph link compression — delta-varint connections (2.4.0 #3)
Cortex registers a graph:compression provider exposing `encode`/`decode`
for HNSW connection lists (delta-varint, ~4× smaller edges than the
JSON-UUID-array shape brainy has persisted since the beginning). This
wires the consumer side without changing the saveHNSWData/getHNSWData
adapter signatures.
Architecture:
- NEW ConnectionsCodec (src/hnsw/connectionsCodec.ts) — translates between
UUID-keyed Map<level, Set<UUID>> and a single compact per-node buffer
via the provider + stable EntityIdMapper. Custom wire format:
[count: u8] [[level: u8] [len: u32 LE] [bytes...]]*. One blob per node
regardless of level count, so the storage I/O is identical to the
legacy path (one saveHNSWData call) plus a single saveBinaryBlob.
- HNSWIndex changes — a `connectionsCodec: ConnectionsCodec | null` field
with `setConnectionsCodec()` setter. THREE save sites (deferred flush,
immediate-mode entity persist, immediate-mode neighbor updates) now go
through a single `persistNodeConnections(nodeId, noun)` helper: when the
codec is wired AND the storage adapter exposes saveBinaryBlob, it
encodes the connections, stores them at `_hnsw_conn/<nodeId>`, and
records `connections: {}` in saveHNSWData as the marker. Otherwise the
legacy JSON-array path is taken. TWO load sites use a matching
`restoreNodeConnections` helper that tries loadBinaryBlob first and
falls back to the legacy hnswData.connections field on miss / decode
error. Format convergence is lazy: pre-2.4.0 nodes still load via the
legacy path, then write the compressed form on next dirty save.
- brainy.ts wireConnectionsCodec — runs alongside wireMmapVectorBackend
during init. Activates when (a) the graph:compression provider is
registered and (b) the metadata index exposes its idMapper. Unlike the
mmap-vector backend, this layer engages on EVERY brainy 7.25.0
adapter — the blob primitive itself is universal; only the codec
presence gates activation.
- Provider interface GraphCompressionProvider in plugin.ts — encode +
decode static signatures. Brainy depends on the interface; cortex's
registered { encode: encodeConnections, decode: decodeConnections }
satisfies it structurally.
Tests (1437 total, +4 vs prior tip):
- tests/unit/hnsw/connections-codec.test.ts — 4 unit tests with a JSON
mock provider: single-level round-trip, empty-Map round-trip (one-byte
buffer), multi-level fan-out round-trip, and silent-drop of UUIDs the
decoding idMapper no longer knows. The real cross-language byte
format is exercised when cortex 2.4.0 wires its delta-varint
encode/decode in.
This completes the brainy half of the 2.4.0 storage foundation: stable
ids (#23), mmap vectors (#24), graph link compression (#25), and
column-store interchange (#26). Coordinated release as brainy 7.26.0 +
cortex 2.4.0 follows.
2026-05-28 10:37:01 -07:00
|
|
|
* and convergence to the compressed form happens lazily on next save.
|
|
|
|
|
*
|
|
|
|
|
* Activated only when the storage adapter exposes the binary-blob primitive
|
|
|
|
|
* AND the metadata index resolves a stable idMapper. Cloud adapters that
|
|
|
|
|
* lack a real local-path resolution still benefit, since the blob primitive
|
|
|
|
|
* itself works across every adapter as of brainy 7.25.0.
|
|
|
|
|
*/
|
|
|
|
|
export interface GraphCompressionProvider {
|
|
|
|
|
/** Encode a list of u32 ints to compact delta-varint bytes. Sorts internally. */
|
|
|
|
|
encode(ids: number[]): Buffer
|
|
|
|
|
/** Decode delta-varint bytes back to a u32 list. */
|
|
|
|
|
decode(data: Buffer): number[]
|
|
|
|
|
}
|
|
|
|
|
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
/**
|
|
|
|
|
* Storage adapter factory — plugins register these to provide
|
|
|
|
|
* new storage backends that users reference by name.
|
|
|
|
|
*
|
|
|
|
|
* Example: A Redis plugin registers 'storage:redis', then users
|
|
|
|
|
* can use `new Brainy({ storage: 'redis', redis: { host: '...' } })`
|
|
|
|
|
*/
|
|
|
|
|
export interface StorageAdapterFactory {
|
|
|
|
|
create(config: Record<string, unknown>): StorageAdapter | Promise<StorageAdapter>
|
|
|
|
|
name: string
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Plugin registry — manages plugin lifecycle and provider resolution.
|
|
|
|
|
*/
|
2026-06-19 10:13:04 -07:00
|
|
|
/** Parse `X.Y.Z[-prerelease][+build]` → `[major, minor, patch]`, prerelease/build stripped. */
|
|
|
|
|
function parseSemverCore(version: string): [number, number, number] | null {
|
|
|
|
|
const core = version.trim().split('+')[0].split('-')[0]
|
|
|
|
|
const parts = core.split('.')
|
|
|
|
|
if (parts.length < 1) return null
|
|
|
|
|
const nums = [0, 1, 2].map((i) => {
|
|
|
|
|
const n = parseInt(parts[i] ?? '0', 10)
|
|
|
|
|
return Number.isFinite(n) ? n : NaN
|
|
|
|
|
})
|
|
|
|
|
if (nums.some((n) => Number.isNaN(n))) return null
|
|
|
|
|
return nums as [number, number, number]
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Compare two semver cores. <0 if a<b, 0 if equal, >0 if a>b. */
|
|
|
|
|
function compareSemverCore(a: [number, number, number], b: [number, number, number]): number {
|
|
|
|
|
for (let i = 0; i < 3; i++) {
|
|
|
|
|
if (a[i] !== b[i]) return a[i] - b[i]
|
|
|
|
|
}
|
|
|
|
|
return 0
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Minimal, dependency-free semver-range check for plugin↔brainy version coupling
|
|
|
|
|
* (see {@link BrainyPlugin.brainyRange}). Brainy is a public MIT library, so we
|
|
|
|
|
* avoid a full `semver` dependency; the coupling we enforce is major-version
|
|
|
|
|
* lockstep with the native accelerator. Supports a space-separated AND of
|
|
|
|
|
* `>=`, `>`, `<=`, `<`, `=` comparators plus `^X.Y.Z` (same-major, >= floor).
|
|
|
|
|
* Prerelease tags on the running `version` are tolerated (compared by core), so
|
|
|
|
|
* an RC build (`8.0.0-rc1`) satisfies `>=8.0.0`.
|
|
|
|
|
*
|
|
|
|
|
* @returns true if `version` satisfies `range`; on an unparseable range, returns
|
|
|
|
|
* true (fail-open on a malformed declaration rather than block a valid brain).
|
|
|
|
|
*/
|
|
|
|
|
export function pluginRangeSatisfies(version: string, range: string): boolean {
|
|
|
|
|
const v = parseSemverCore(version)
|
|
|
|
|
if (!v) return false
|
|
|
|
|
const comparators = range.trim().split(/\s+/).filter(Boolean)
|
|
|
|
|
if (comparators.length === 0) return true
|
|
|
|
|
for (const comp of comparators) {
|
|
|
|
|
const m = comp.match(/^(\^|>=|<=|>|<|=)?\s*(\d+(?:\.\d+){0,2}(?:[-+].*)?)$/)
|
|
|
|
|
if (!m) return true // unparseable comparator → fail-open
|
|
|
|
|
const op = m[1] || '='
|
|
|
|
|
const target = parseSemverCore(m[2])
|
|
|
|
|
if (!target) return true
|
|
|
|
|
const cmp = compareSemverCore(v, target)
|
|
|
|
|
let ok: boolean
|
|
|
|
|
switch (op) {
|
|
|
|
|
case '>=': ok = cmp >= 0; break
|
|
|
|
|
case '>': ok = cmp > 0; break
|
|
|
|
|
case '<=': ok = cmp <= 0; break
|
|
|
|
|
case '<': ok = cmp < 0; break
|
|
|
|
|
case '^': ok = v[0] === target[0] && cmp >= 0; break // same major, >= floor
|
|
|
|
|
default: ok = cmp === 0 // '='
|
|
|
|
|
}
|
|
|
|
|
if (!ok) return false
|
|
|
|
|
}
|
|
|
|
|
return true
|
|
|
|
|
}
|
|
|
|
|
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
export class PluginRegistry {
|
|
|
|
|
private plugins: Map<string, BrainyPlugin> = new Map()
|
|
|
|
|
private providers: Map<string, unknown> = new Map()
|
|
|
|
|
private activated: Set<string> = new Set()
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Register a plugin manually.
|
|
|
|
|
*/
|
|
|
|
|
register(plugin: BrainyPlugin): void {
|
|
|
|
|
this.plugins.set(plugin.name, plugin)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Activate all registered plugins.
|
|
|
|
|
*/
|
|
|
|
|
async activateAll(context: BrainyPluginContext): Promise<string[]> {
|
|
|
|
|
const activated: string[] = []
|
|
|
|
|
|
|
|
|
|
for (const [name, plugin] of this.plugins) {
|
|
|
|
|
if (this.activated.has(name)) continue
|
|
|
|
|
|
2026-06-19 10:13:04 -07:00
|
|
|
// Version-coupling guard. A registered plugin is ALWAYS explicitly
|
|
|
|
|
// requested (config.plugins or brain.use() — brainy does no
|
|
|
|
|
// auto-detection), so a version mismatch must FAIL LOUD, never silently
|
|
|
|
|
// degrade to the default JS engine. (This was the #1 cross-repo drift:
|
|
|
|
|
// an old @soulcraft/cor running invisibly against brainy 8.x.)
|
|
|
|
|
if (plugin.brainyRange && !pluginRangeSatisfies(context.version, plugin.brainyRange)) {
|
|
|
|
|
throw new Error(
|
|
|
|
|
`[brainy] Plugin "${name}" supports brainy ${plugin.brainyRange}, but this is brainy ` +
|
|
|
|
|
`${context.version}. The engines must be version-matched (brainy 8.x ↔ @soulcraft/cor 3.x); ` +
|
|
|
|
|
`install a compatible ${name} (or brainy). brainy will NOT silently run the default engine ` +
|
|
|
|
|
`in place of a mismatched accelerator.`
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let success: boolean
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
try {
|
2026-06-19 10:13:04 -07:00
|
|
|
success = await plugin.activate(context)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
} catch (error) {
|
2026-06-19 10:13:04 -07:00
|
|
|
// Hard activation failure on an explicitly-requested plugin is fatal,
|
|
|
|
|
// not a swallow-and-fall-back-to-JS.
|
|
|
|
|
throw new Error(
|
|
|
|
|
`[brainy] Plugin "${name}" failed to activate: ` +
|
|
|
|
|
`${error instanceof Error ? error.message : String(error)}`
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (success) {
|
|
|
|
|
this.activated.add(name)
|
|
|
|
|
activated.push(name)
|
|
|
|
|
} else {
|
|
|
|
|
// Documented graceful decline (activate() → false). Surface it loudly so
|
|
|
|
|
// a silent degrade to the default engine never goes unnoticed.
|
|
|
|
|
console.warn(
|
|
|
|
|
`[brainy] Plugin "${name}" declined activation (activate() returned false); ` +
|
|
|
|
|
`the default engine is in use for its providers.`
|
|
|
|
|
)
|
feat: add plugin system for cortex and storage adapters
Simple plugin architecture with two use cases:
1. Native acceleration (@soulcraft/brainy-cortex) — auto-detected
2. Custom storage adapters (e.g., Redis, DynamoDB)
Plugins implement BrainyPlugin interface with activate/deactivate lifecycle.
Auto-detection tries dynamic import of known packages during init().
Manual registration via brain.use(plugin) before init().
Provider keys: metadataIndex, graphIndex, entityIdMapper, cache, hnsw,
roaring, embeddings, distance, msgpack, storage:<name>
2026-01-31 09:22:32 -08:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return activated
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Deactivate all plugins (called during close()).
|
|
|
|
|
*/
|
|
|
|
|
async deactivateAll(): Promise<void> {
|
|
|
|
|
for (const [name, plugin] of this.plugins) {
|
|
|
|
|
if (!this.activated.has(name)) continue
|
|
|
|
|
try {
|
|
|
|
|
await plugin.deactivate?.()
|
|
|
|
|
this.activated.delete(name)
|
|
|
|
|
} catch {
|
|
|
|
|
// Non-fatal
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a registered provider by key.
|
|
|
|
|
*/
|
|
|
|
|
getProvider<T = unknown>(key: string): T | undefined {
|
|
|
|
|
return this.providers.get(key) as T | undefined
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Check if a provider is registered.
|
|
|
|
|
*/
|
|
|
|
|
hasProvider(key: string): boolean {
|
|
|
|
|
return this.providers.has(key)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Register a provider (called by plugins via BrainyPluginContext).
|
|
|
|
|
*/
|
|
|
|
|
registerProvider(key: string, implementation: unknown): void {
|
|
|
|
|
this.providers.set(key, implementation)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a storage adapter factory by name.
|
|
|
|
|
*/
|
|
|
|
|
getStorageFactory(name: string): StorageAdapterFactory | undefined {
|
|
|
|
|
return this.providers.get(`storage:${name}`) as StorageAdapterFactory | undefined
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Get active plugin names */
|
|
|
|
|
getActivePlugins(): string[] {
|
|
|
|
|
return [...this.activated]
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Check if any plugins are active */
|
|
|
|
|
hasActivePlugins(): boolean {
|
|
|
|
|
return this.activated.size > 0
|
|
|
|
|
}
|
|
|
|
|
}
|