soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat(8.0): brain.graph.subgraph(query) query→expand fusion (#61)

Browse source 
The subgraph seed selector now accepts not just entity id(s) but a find() result
or a FindParams query — brain.graph.subgraph({ where: { team: 'platform' } },
{ depth: 1 }) runs the query and expands the neighborhood of every match in one
call. Existing id-seeded calls are unchanged.

The win is the native query→expand path: for a metadata-only query, the matched
universe is forwarded to traverse() as an OpaqueIdSet (a roaring Buffer) with NO
id materialization in TypeScript — the find() result never leaves the engine's
representation (the O(1)-crossing the cor 3.0 contract is built around). The
pure-JS path (or any query carrying vector/text/proximity criteria) materializes
the matched ids via find() and seeds the traversal from them, capped at a bounded
default when the caller pins no limit.

- New public `SubgraphSelector<T>` union (id | id[] | Result[] | FindParams).
- find()'s metadata filter-builder extracted to a shared `buildMetadataFilter`
  (reused by the opaque universe producer); behavior unchanged.
- graphSubgraphNative generalized to accept `bigint[] | OpaqueIdSet` seeds.

Tested: JS query→expand + Result[] selector + empty-match in graph-subgraph, and
the native opaque pass-through (Buffer reaches traverse unmaterialized) vs the
find()-materialized bigint[] seeds via the mock provider in graph-native-routing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-23 13:30:30 -07:00
parent dd325f2f94
commit 82dde92077
5 changed files with 233 additions and 57 deletions
Download patch file Download diff file Expand all files Collapse all files

7
RELEASES.md
View file

@ -73,6 +73,13 @@ no separate migration doc. **`8.0.0-rc.2` is live** — install with
> matches that DO pass the filter, rather than coming back empty. No API change. With the native
> `@soulcraft/cor` 3.0 stack the matched universe is forwarded as an opaque roaring buffer with
> zero id materialization in TypeScript; the pure-JS path restricts the beam walk with a string set.
> - **`brain.graph.subgraph()` accepts a query (query→expand).** The seed selector now takes not
> just entity id(s) but a `find()` result or a `FindParams` query — `brain.graph.subgraph({ where:
> { team: 'platform' } }, { depth: 1 })` runs the query and expands the neighborhood of every
> match in one call. With the native engine a metadata-only query's matched universe is handed to
> the traversal as an opaque set with no id materialization in TypeScript (the query→expand
> fusion); the pure-JS path materializes the matched ids and expands from them. Existing
> id-seeded calls are unchanged.
### Headline: Database as a Value

192
src/brainy.ts
View file

@ -108,6 +108,7 @@ import {
GraphView,
GraphNode,
SubgraphOptions,
SubgraphSelector,
GraphExportOptions,
GraphRankOptions,
GraphRankEntry,
@ -364,6 +365,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
/** Cap for {@link Brainy.verbIntWarmCache} (~100k entries ≈ a few MB). */
private static readonly VERB_INT_WARM_CACHE_MAX = 100_000
/**
* Default cap on how many `find()` matches seed a `graph.subgraph(query)` when the
* caller pins no explicit `limit` — bounds the materialized seed set on the JS /
* non-opaque path (the native opaque path forwards the whole universe, uncapped).
*/
private static readonly QUERY_SEED_CAP = 10_000
// Silent mode state
private originalConsole?: {
log: typeof console.log
@ -3278,8 +3286,8 @@ export class Brainy<T = any> implements BrainyInterface<T> {
*/
get graph(): GraphApi<T> {
return {
subgraph: (seeds: string | string[], options?: SubgraphOptions) =>
this.graphSubgraph(seeds, options),
subgraph: (selector: SubgraphSelector<T>, options?: SubgraphOptions) =>
this.graphSubgraph(selector, options),
export: (options?: GraphExportOptions) => this.graphExport(options),
rank: (options?: GraphRankOptions) => this.graphRank(options),
communities: (options?: GraphCommunitiesOptions) => this.graphCommunities(options),
@ -3313,24 +3321,96 @@ export class Brainy<T = any> implements BrainyInterface<T> {
/**
* @description {@link GraphApi.subgraph} dispatch: native engine when present,
* else the TS BFS fallback. Seeds are id-normalized so a caller may seed by
* natural key.
* else the TS BFS fallback. The selector is an id set, a `find()` result, or a
* query (query→expand fusion). Explicit ids are id-normalized so a caller may
* seed by natural key.
*/
private async graphSubgraph(
seeds: string | string[],
selector: SubgraphSelector<T>,
options?: SubgraphOptions
): Promise<GraphView<T>> {
await this.ensureInitialized()
const seedIds = (Array.isArray(seeds) ? seeds : [seeds]).map((s) => resolveEntityId(s))
const depth = Math.max(0, options?.depth ?? 1)
const direction = options?.direction ?? 'both'
const accel = this.graphAccelerationProvider()
// Query selector (a FindParams object — not a string / array) → query→expand.
if (selector && typeof selector === 'object' && !Array.isArray(selector)) {
return this.graphSubgraphFromQuery(accel, selector, depth, direction, options)
}
// Id set: a string, an array of ids, or a `find()` result (Result[]). The
// FindParams (query) case returned above, so the remaining union is the id forms.
const seedIds = this.selectorToSeedIds(selector as string | string[] | Result<T>[])
if (seedIds.length === 0) return { nodes: [], edges: [], truncated: false }
return accel
? this.graphSubgraphNative(accel, seedIds, depth, direction, options)
? this.graphSubgraphNative(accel, this.seedIdsToInts(seedIds), depth, direction, options)
: this.graphSubgraphFallback(seedIds, depth, direction, options)
}
/**
* @description Query→expand fusion (graph #61): run the query, then expand the
* subgraph from every match. When the native engine is present AND the query is a
* pure metadata filter (no vector/text/proximity criteria) AND the metadata
* provider exposes the opaque-set producer, the matched universe is forwarded to
* `traverse` as an {@link OpaqueIdSet} with NO id materialization in TS — the
* O(1)-crossing path. Otherwise the query is materialized via `find()` and its
* result ids seed the traversal (native or TS fallback).
*/
private async graphSubgraphFromQuery(
accel: GraphAccelerationProvider | undefined,
selector: FindParams<T>,
depth: number,
direction: 'in' | 'out' | 'both',
options?: SubgraphOptions
): Promise<GraphView<T>> {
// Native opaque fast path: a metadata-only universe never leaves the engine.
if (accel && !this.hasVectorOrTextCriteria(selector)) {
const filter = this.buildMetadataFilter(selector)
const mip = this.metadataIndex as unknown as MetadataIndexProvider
if (filter && typeof mip.getIdSetForFilter === 'function') {
const universe = await mip.getIdSetForFilter(filter)
return this.graphSubgraphNative(accel, universe, depth, direction, options)
}
}
// General path: materialize the matched ids, then expand from them. The find
// default limit (10) is too small to seed a neighborhood, so seed from ALL
// matches up to a bounded cap unless the caller pinned an explicit limit.
const matches = await this.find({
...selector,
limit: selector.limit ?? Brainy.QUERY_SEED_CAP
})
const seedIds = matches.map((m) => m.id)
if (seedIds.length === 0) return { nodes: [], edges: [], truncated: false }
return accel
? this.graphSubgraphNative(accel, this.seedIdsToInts(seedIds), depth, direction, options)
: this.graphSubgraphFallback(seedIds, depth, direction, options)
}
/** True when a query needs vector/text/proximity search (not a pure metadata filter). */
private hasVectorOrTextCriteria(params: FindParams<T>): boolean {
return Boolean(
(params.query && params.query.trim() !== '') || params.vector || params.near
)
}
/** Normalize an id-set selector (id / id[] / `find()` result) to canonical seed ids. */
private selectorToSeedIds(selector: string | string[] | Result<T>[]): string[] {
const arr = Array.isArray(selector) ? selector : [selector]
return arr.map((s) => resolveEntityId(typeof s === 'string' ? s : s.id))
}
/** Resolve seed ids to graph entity ints, dropping any that were never mapped. */
private seedIdsToInts(seedIds: string[]): bigint[] {
const ints: bigint[] = []
for (const id of seedIds) {
const int = this.graphEntityInt(id)
if (int !== undefined) ints.push(int)
}
return ints
}
/**
* @description Pure-TS subgraph BFS — expands each frontier through the
* O(degree) `related()` adjacency (visibility / type / subtype filters applied
@ -3413,17 +3493,14 @@ export class Brainy<T = any> implements BrainyInterface<T> {
*/
private async graphSubgraphNative(
accel: GraphAccelerationProvider,
seedIds: string[],
seeds: bigint[] | OpaqueIdSet,
depth: number,
direction: 'in' | 'out' | 'both',
options?: SubgraphOptions
): Promise<GraphView<T>> {
const seedInts: bigint[] = []
for (const id of seedIds) {
const int = this.graphEntityInt(id)
if (int !== undefined) seedInts.push(int)
}
if (seedInts.length === 0) return { nodes: [], edges: [], truncated: false }
// Resolved int seeds may be empty (no mapped entities); an OpaqueIdSet (Buffer)
// is passed straight through — the provider owns its membership.
if (Array.isArray(seeds) && seeds.length === 0) return { nodes: [], edges: [], truncated: false }
const verbTypes = options?.type
? (Array.isArray(options.type) ? options.type : [options.type]).map((t) =>
@ -3445,7 +3522,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
includeDepth: true
}
const sub = await accel.traverse(seedInts, traverseOptions)
const sub = await accel.traverse(seeds, traverseOptions)
return this.hydrateNativeSubgraph(sub, options?.hydrateNodes !== false)
}
@ -4816,40 +4893,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
let preResolvedAllowedIds: OpaqueIdSet | undefined
if (params.where || params.type || params.subtype || params.service || params.excludeVFS) {
preResolvedFilter = {}
if (params.where) {
Object.assign(preResolvedFilter, params.where)
// Alias: where.type → where.noun (storage field name for entity type)
if ('type' in preResolvedFilter && !('noun' in preResolvedFilter)) {
preResolvedFilter.noun = preResolvedFilter.type
delete preResolvedFilter.type
}
}
if (params.service) preResolvedFilter.service = params.service
if (params.excludeVFS === true) {
preResolvedFilter.vfsType = { exists: false }
preResolvedFilter.isVFSEntity = { ne: true }
}
// Subtype (top-level standard field — fast path).
// Must be assigned BEFORE the type-array expansion below.
if (params.subtype !== undefined) {
preResolvedFilter.subtype = Array.isArray(params.subtype)
? { oneOf: params.subtype }
: params.subtype
}
if (params.type) {
const types = Array.isArray(params.type) ? params.type : [params.type]
if (types.length === 1) {
preResolvedFilter.noun = types[0]
} else {
preResolvedFilter = {
anyOf: types.map(type => ({
noun: type,
...preResolvedFilter
}))
}
}
}
preResolvedFilter = this.buildMetadataFilter(params)
preResolvedMetadataIds = await this.metadataIndex.getIdsForFilter(preResolvedFilter)
// Visibility hard filter — restrict the HNSW candidate set to non-hidden ids.
@ -11145,6 +11189,54 @@ export class Brainy<T = any> implements BrainyInterface<T> {
return params
}
/**
* @description Build the MetadataIndex filter object from a query's structured
* criteria (`where` / `type` / `subtype` / `service` / `excludeVFS`) — the shared
* filter shape consumed by `getIdsForFilter` / `getIdSetForFilter`. Applies the
* `where.type → noun` alias and expands a multi-type filter into an `anyOf`.
* @param params - The structured query criteria.
* @returns The filter object, or `null` when no structured criteria are present.
*/
private buildMetadataFilter(params: {
where?: any
type?: NounType | NounType[]
subtype?: string | string[]
service?: string
excludeVFS?: boolean
}): any | null {
if (!(params.where || params.type || params.subtype || params.service || params.excludeVFS)) {
return null
}
let filter: any = {}
if (params.where) {
Object.assign(filter, params.where)
// Alias: where.type → where.noun (storage field name for entity type)
if ('type' in filter && !('noun' in filter)) {
filter.noun = filter.type
delete filter.type
}
}
if (params.service) filter.service = params.service
if (params.excludeVFS === true) {
filter.vfsType = { exists: false }
filter.isVFSEntity = { ne: true }
}
// Subtype (top-level standard field — fast path). Assigned BEFORE the type-array
// expansion below so the spread into each anyOf branch carries it through.
if (params.subtype !== undefined) {
filter.subtype = Array.isArray(params.subtype) ? { oneOf: params.subtype } : params.subtype
}
if (params.type) {
const types = Array.isArray(params.type) ? params.type : [params.type]
if (types.length === 1) {
filter.noun = types[0]
} else {
filter = { anyOf: types.map((type) => ({ noun: type, ...filter })) }
}
}
return filter
}
/**
* Execute vector search component
*

27
src/types/brainy.types.ts
View file

@ -913,6 +913,17 @@ export interface GraphPathResult {
cost: number
}
/**
* What {@link GraphApi.subgraph} expands from — "the things to grow a neighborhood
* around". One of:
* - an entity id, or an array of them (the explicit id set),
* - a {@link Result} array (a `find()` result — its entities become the seeds), or
* - a {@link FindParams} query (the query→expand fusion: run the query, expand from
* every match). With the native engine a metadata-only query's matched universe is
* forwarded to the traversal as an opaque set with no id materialization.
*/
export type SubgraphSelector<T = any> = string | string[] | Result<T>[] | FindParams<T>
/**
* The `brain.graph` namespace — graph-shaped reads over the knowledge graph.
* Routes to a native {@link import('../plugin.js').GraphAccelerationProvider}
@ -921,17 +932,23 @@ export interface GraphPathResult {
*/
export interface GraphApi<T = any> {
/**
* @description Extract the subgraph reachable from one or more seed entities —
* the bounded multi-hop neighborhood, as nodes + edges. The one-call answer to
* "show me everything around this node, N hops out".
* @param seeds - A seed entity id, or an array of them.
* @description Extract the subgraph reachable from a seed selector — the bounded
* multi-hop neighborhood, as nodes + edges. The one-call answer to "show me
* everything around this, N hops out". The seed can be entity id(s), a `find()`
* result, or a {@link FindParams} query (query→expand: run the query, then expand
* from every match — with the native engine the matched universe never leaves
* the engine's representation).
* @param selector - Seed id(s), a `find()` result, or a query. See {@link SubgraphSelector}.
* @param options - Depth, direction, edge/visibility filters, and caps.
* @returns The hydrated {@link GraphView}.
* @example
* const view = await brain.graph.subgraph(personId, { depth: 2 })
* // view.nodes = the 2-hop neighborhood; view.edges = the relations among them
* @example
* // Query→expand: the neighborhood of everything matching a filter.
* const cluster = await brain.graph.subgraph({ where: { team: 'platform' } }, { depth: 1 })
*/
subgraph(seeds: string | string[], options?: SubgraphOptions): Promise<GraphView<T>>
subgraph(selector: SubgraphSelector<T>, options?: SubgraphOptions): Promise<GraphView<T>>
/**
* @description Stream the WHOLE graph in one O(N+E) pass as a sequence of

33
tests/unit/brainy/graph-native-routing.test.ts
View file

@ -27,13 +27,15 @@ function makeMockAccel() {
rankResult: any
communitiesResult: any
pathResult: any
lastTraverseSeeds: any
} = {
calls,
traverseResult: null,
cursorChunks: [],
rankResult: null,
communitiesResult: null,
pathResult: null
pathResult: null,
lastTraverseSeeds: undefined
}
const empty = { nodeInts: new BigInt64Array(0), scores: new Float64Array(0) }
const emptyCommunities = {
@ -43,8 +45,9 @@ function makeMockAccel() {
}
const provider = {
isInitialized: true,
traverse: async () => {
traverse: async (seeds: any) => {
calls.traverse++
state.lastTraverseSeeds = seeds
return state.traverseResult
},
edgesForNode: async () => state.traverseResult,
@ -261,4 +264,30 @@ describe('brain.graph.* native routing + columnar hydration (native seam)', () =
expect(await brain.graph.path(a, c)).toBeNull()
expect(mock.state.calls.path).toBe(1)
})
it('subgraph(query) forwards the metadata universe to traverse as an OpaqueIdSet (query→expand #61)', async () => {
// The native metadata index would return its roaring filter result as a Buffer;
// stub that producer and assert it reaches traverse WITHOUT id materialization.
const sentinel = new Uint8Array([0x01, 0x02, 0x03])
;(brain as any).metadataIndex.getIdSetForFilter = async () => sentinel
mock.state.traverseResult = await realSubgraph()
await brain.graph.subgraph({ type: NounType.Person }, { depth: 1 })
expect(mock.state.calls.traverse).toBe(1)
// The opaque Buffer is the traverse seed argument — passed straight through.
expect(mock.state.lastTraverseSeeds).toBe(sentinel)
})
it('subgraph(query) materializes seeds via find() when no opaque producer exists', async () => {
// No getIdSetForFilter on the (real) metadata index → general path: find() runs,
// its matched ids resolve to entity ints, and those seed the native traverse.
mock.state.traverseResult = await realSubgraph()
await brain.graph.subgraph({ type: NounType.Person }, { depth: 1 })
expect(mock.state.calls.traverse).toBe(1)
expect(Array.isArray(mock.state.lastTraverseSeeds)).toBe(true) // bigint[], not a Buffer
expect(typeof (mock.state.lastTraverseSeeds as unknown[])[0]).toBe('bigint')
})
})

31
tests/unit/brainy/graph-subgraph.test.ts
View file

@ -98,4 +98,35 @@ describe('brain.graph.subgraph() (graph engine #25)', () => {
// depth 0 = just the seeds, no expansion.
expect(new Set(view.nodes.map((n) => n.id))).toEqual(new Set([a, c]))
})
// ---- Query→expand fusion (graph #61): seed from a query / find() result ----
it('query→expand: subgraph(query) seeds from the matches and expands their neighborhood', async () => {
// The only Project is d; 1-hop expansion brings in a (a → d ReportsTo).
const view = await brain.graph.subgraph({ type: NounType.Project }, { depth: 1 })
const ids = new Set(view.nodes.map((n) => n.id))
expect(ids.has(d)).toBe(true) // the query match (seed)
expect(ids.has(a)).toBe(true) // expanded neighbor
expect(view.edges.some((r) => r.from === a && r.to === d)).toBe(true)
})
it('query→expand: depth 0 returns just the matches, no expansion', async () => {
const view = await brain.graph.subgraph({ type: NounType.Project }, { depth: 0 })
expect(new Set(view.nodes.map((n) => n.id))).toEqual(new Set([d]))
})
it('accepts a find() result array as the seed set', async () => {
const matches = await brain.find({ type: NounType.Project }) // → [d]
expect(matches.length).toBeGreaterThan(0)
const view = await brain.graph.subgraph(matches, { depth: 1 })
const ids = new Set(view.nodes.map((n) => n.id))
expect(ids.has(d)).toBe(true)
expect(ids.has(a)).toBe(true)
})
it('query→expand returns empty when the query matches nothing', async () => {
const view = await brain.graph.subgraph({ type: NounType.Event }, { depth: 1 })
expect(view.nodes).toEqual([])
expect(view.edges).toEqual([])
})
})