feat(8.0): u64 BigInt graph provider contract — punch list a-d,g,h
GraphIndexProvider now speaks BigInt at the boundary (D.2 mirror): - getNeighbors/getVerbIdsBySource/getVerbIdsByTarget take entity ints and return entity/verb ints as bigint[] - new REQUIRED verbIntsToIds(bigint[]) batch reverse resolver (L.7 identity-fingerprint design — verb ids are UUIDs by contract, so the provider-side interning is losslessly reversible) - addVerb(verb, sourceInt, targetInt) returns the interned verb int; removeVerb(verbId) joins the contract Coordinator (brainy.ts) owns ALL UUID <-> int conversion: getOrAssign on writes, getInt on reads (unmapped UUID -> empty result without calling the provider), getUuid / verbIntsToIds on returns, plus a bounded ~100k-entry insertion-order warm cache for verb-int -> verb-id pairs fed by addVerb returns and resolver results. GraphVerb gains derived sourceInt/targetInt (populated at add time, never persisted). findConnectedSubtype gains a native fast path that routes single-type single-subtype outgoing BFS through the provider when available. JS GraphAdjacencyIndex satisfies the contract while staying string/u32-keyed internally: entity ints resolve through the shared entity-id mapper (threaded in by the coordinator on init/fork/checkout), verb ints come from an in-process append-only interning map re-derived from storage on rebuild/cold-start. ColumnStoreProvider widens the same way: addEntity/ removeEntity take bigint, sortTopK/filteredSortTopK return bigint[]. relate() now rejects a caller-supplied id with a teaching error — verb ids are brainy-generated UUIDs by contract in 8.0 (previously a passed id was silently ignored). No Roaring64 provider-boundary decode site exists yet; the JS-internal column store stays Roaring32 and the Treemap decoder lands with the first consumer of provider-returned filter buffers. Public brain API unchanged. 1413 tests green (+10 new BigInt contract tests).
This commit is contained in:
parent
62f6472fa0
commit
2427bb7960
17 changed files with 1164 additions and 328 deletions
|
|
@ -1,11 +1,20 @@
|
|||
/**
|
||||
* GraphAdjacencyIndex - Billion-Scale Graph Traversal Engine
|
||||
* @module graph/graphAdjacencyIndex
|
||||
* @description GraphAdjacencyIndex — billion-scale graph traversal engine.
|
||||
*
|
||||
* NOW SCALES TO BILLIONS: LSM-tree storage reduces memory from 500GB to 1.3GB
|
||||
* for 1 billion relationships while maintaining sub-5ms neighbor lookups.
|
||||
* LSM-tree storage reduces memory from 500GB to 1.3GB for 1 billion
|
||||
* relationships while maintaining sub-5ms neighbor lookups.
|
||||
*
|
||||
* NO FALLBACKS - NO MOCKS - REAL PRODUCTION CODE
|
||||
* Handles billions of relationships with sustainable memory usage
|
||||
* **8.0 u64 boundary:** this class implements the BigInt
|
||||
* {@link GraphIndexProvider} contract — entity ints in, entity/verb ints out.
|
||||
* Internally everything stays string-keyed (UUID-keyed LSM trees) and u32:
|
||||
* entity-int params resolve to UUIDs via the shared entity-id mapper
|
||||
* (`getUuid(Number(big))` — lossless under the `EntityIdSpaceExceeded` u32
|
||||
* guard), and returns convert back with `BigInt(getOrAssign(uuid))`. Verb ints
|
||||
* come from a small in-process interning map (see
|
||||
* {@link GraphAdjacencyIndex.verbIntsToIds}); the interning is derived state,
|
||||
* never persisted — `rebuild()` / the verb-id-set recovery path re-derive it
|
||||
* from storage, which the JS index already does on cold start.
|
||||
*/
|
||||
|
||||
import { GraphVerb, StorageAdapter } from '../coreTypes.js'
|
||||
|
|
@ -21,6 +30,22 @@ export interface GraphIndexConfig {
|
|||
flushInterval?: number // Default: 30000ms
|
||||
}
|
||||
|
||||
/**
|
||||
* @description The minimal UUID ↔ int resolver surface the JS graph index
|
||||
* needs at its BigInt boundary. Satisfied by `EntityIdMapper` (and by whatever
|
||||
* `MetadataIndexManager.getIdMapper()` / a native metadata index returns) —
|
||||
* declared structurally here so the graph layer doesn't import the metadata
|
||||
* layer.
|
||||
*/
|
||||
export interface GraphEntityIdResolver {
|
||||
/** Resolve a UUID to its int, assigning a new one if absent (write path). */
|
||||
getOrAssign(uuid: string): number
|
||||
/** Resolve a UUID to its int without assigning (read path). */
|
||||
getInt(uuid: string): number | undefined
|
||||
/** Reverse-resolve an int to its UUID (`undefined` = unknown/deleted). */
|
||||
getUuid(intId: number): string | undefined
|
||||
}
|
||||
|
||||
export interface GraphIndexStats {
|
||||
totalRelationships: number
|
||||
sourceNodes: number
|
||||
|
|
@ -51,6 +76,18 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
// Now: Set<string> stores only IDs (~100KB @ 1B verbs) = 1,280,000x reduction
|
||||
private verbIdSet = new Set<string>()
|
||||
|
||||
// Verb-id interning for the BigInt boundary (8.0 u64 contract).
|
||||
// Process-lifetime derived state: assigned on addVerb / rebuild /
|
||||
// verb-id-set recovery, NEVER persisted. removeVerb keeps the entry so a
|
||||
// verb int stays stable (and resolvable) for the index's lifetime.
|
||||
private verbIdToInt = new Map<string, number>()
|
||||
private verbIntToId: string[] = []
|
||||
|
||||
// Shared UUID ↔ int resolver for entity ints at the BigInt boundary.
|
||||
// Threaded in by the coordinator (brainy.ts) from the metadata index's
|
||||
// idMapper — see setEntityIdMapper().
|
||||
private entityIdMapper?: GraphEntityIdResolver
|
||||
|
||||
// Infrastructure integration
|
||||
private storage: StorageAdapter
|
||||
private unifiedCache: UnifiedCache
|
||||
|
|
@ -75,8 +112,13 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
return this.initialized
|
||||
}
|
||||
|
||||
constructor(storage: StorageAdapter, config: GraphIndexConfig = {}) {
|
||||
constructor(
|
||||
storage: StorageAdapter,
|
||||
config: GraphIndexConfig = {},
|
||||
entityIdMapper?: GraphEntityIdResolver
|
||||
) {
|
||||
this.storage = storage
|
||||
this.entityIdMapper = entityIdMapper
|
||||
this.config = {
|
||||
maxIndexSize: config.maxIndexSize ?? 100000,
|
||||
rebuildThreshold: config.rebuildThreshold ?? 0.1,
|
||||
|
|
@ -116,6 +158,49 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
prodLog.info('GraphAdjacencyIndex initialized with LSM-tree storage (4 LSM-trees total)')
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Thread in the shared UUID ↔ int resolver used for entity-int
|
||||
* conversion at the BigInt boundary. The coordinator (`brainy.ts`) calls this
|
||||
* with `metadataIndex.getIdMapper()` right after index construction (init,
|
||||
* fork, and checkout paths) so reads/writes share one int universe with the
|
||||
* metadata index. Idempotent; safe to call again after a branch switch.
|
||||
* @param mapper - The shared entity-id resolver.
|
||||
* @returns Nothing.
|
||||
*/
|
||||
setEntityIdMapper(mapper: GraphEntityIdResolver): void {
|
||||
this.entityIdMapper = mapper
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the entity-id mapper or fail loudly. The BigInt read methods are
|
||||
* meaningless without a shared int universe — a missing mapper is a wiring
|
||||
* bug, not a recoverable condition.
|
||||
*/
|
||||
private requireEntityIdMapper(): GraphEntityIdResolver {
|
||||
if (!this.entityIdMapper) {
|
||||
throw new Error(
|
||||
'GraphAdjacencyIndex: entityIdMapper not wired. The coordinator must ' +
|
||||
'call setEntityIdMapper(metadataIndex.getIdMapper()) (or pass it to ' +
|
||||
'the constructor) before BigInt-boundary reads.'
|
||||
)
|
||||
}
|
||||
return this.entityIdMapper
|
||||
}
|
||||
|
||||
/**
|
||||
* Intern a verb-id string, assigning the next sequential u32 on first sight.
|
||||
* Append-only for the index's lifetime — removeVerb keeps the entry so verb
|
||||
* ints handed to callers stay resolvable.
|
||||
*/
|
||||
private internVerbId(verbId: string): number {
|
||||
const existing = this.verbIdToInt.get(verbId)
|
||||
if (existing !== undefined) return existing
|
||||
const next = this.verbIntToId.length
|
||||
this.verbIdToInt.set(verbId, next)
|
||||
this.verbIntToId.push(verbId)
|
||||
return next
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize the graph index (lazy initialization)
|
||||
* Added defensive auto-rebuild check for verbIdSet consistency
|
||||
|
|
@ -172,6 +257,10 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
|
||||
for (const verb of result.items) {
|
||||
this.verbIdSet.add(verb.id)
|
||||
// Re-derive the verb-int interning (process-lifetime state, never
|
||||
// persisted — this recovery path is one of the two cold-start sources,
|
||||
// alongside rebuild()).
|
||||
this.internVerbId(verb.id)
|
||||
// Also update counts
|
||||
const verbType = verb.verb || 'unknown'
|
||||
this.relationshipCountsByType.set(
|
||||
|
|
@ -190,49 +279,60 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
}
|
||||
|
||||
/**
|
||||
* Core API - Neighbor lookup with LSM-tree storage
|
||||
* @description Core API — neighbor lookup with LSM-tree storage (BigInt
|
||||
* boundary). O(log n) with bloom filter optimization (90% of queries skip
|
||||
* disk I/O); pagination support for high-degree nodes. The entity int is
|
||||
* resolved to a UUID via the shared mapper (unknown int → empty result) and
|
||||
* neighbor UUIDs convert back via `BigInt(getOrAssign(uuid))`.
|
||||
*
|
||||
* O(log n) with bloom filter optimization (90% of queries skip disk I/O)
|
||||
* Added pagination support for high-degree nodes
|
||||
*
|
||||
* @param id Entity ID to get neighbors for
|
||||
* @param optionsOrDirection Optional: direction string OR options object
|
||||
* @returns Array of neighbor IDs (paginated if limit/offset specified)
|
||||
* @param id - Entity int to get neighbors for (from the shared idMapper).
|
||||
* @param options - Optional direction ('both' default) + limit/offset.
|
||||
* @returns Neighbor entity ints (paginated if limit/offset specified).
|
||||
*
|
||||
* @example
|
||||
* // Get all neighbors (backward compatible)
|
||||
* const all = await graphIndex.getNeighbors(id)
|
||||
* // Get all neighbors of an entity int
|
||||
* const all = await graphIndex.getNeighbors(42n)
|
||||
*
|
||||
* @example
|
||||
* // Get outgoing neighbors (backward compatible)
|
||||
* const out = await graphIndex.getNeighbors(id, 'out')
|
||||
*
|
||||
* @example
|
||||
* // Get first 50 outgoing neighbors (new API)
|
||||
* const page1 = await graphIndex.getNeighbors(id, { direction: 'out', limit: 50 })
|
||||
*
|
||||
* @example
|
||||
* // Paginate through neighbors
|
||||
* const page1 = await graphIndex.getNeighbors(id, { limit: 100, offset: 0 })
|
||||
* const page2 = await graphIndex.getNeighbors(id, { limit: 100, offset: 100 })
|
||||
* // Get first 50 outgoing neighbors
|
||||
* const page1 = await graphIndex.getNeighbors(42n, { direction: 'out', limit: 50 })
|
||||
*/
|
||||
async getNeighbors(
|
||||
id: string,
|
||||
optionsOrDirection?: {
|
||||
id: bigint,
|
||||
options?: {
|
||||
direction?: 'in' | 'out' | 'both'
|
||||
limit?: number
|
||||
offset?: number
|
||||
} | 'in' | 'out' | 'both'
|
||||
): Promise<string[]> {
|
||||
}
|
||||
): Promise<bigint[]> {
|
||||
await this.ensureInitialized()
|
||||
|
||||
// Normalize old API (direction string) to new API (options object)
|
||||
const options = typeof optionsOrDirection === 'string'
|
||||
? { direction: optionsOrDirection }
|
||||
: (optionsOrDirection || {})
|
||||
const mapper = this.requireEntityIdMapper()
|
||||
const uuid = mapper.getUuid(Number(id))
|
||||
if (uuid === undefined) {
|
||||
// Unknown/deleted entity int — no edges by definition.
|
||||
return []
|
||||
}
|
||||
|
||||
const neighborUuids = await this.getNeighborUuids(uuid, options)
|
||||
return neighborUuids.map(neighborUuid => BigInt(mapper.getOrAssign(neighborUuid)))
|
||||
}
|
||||
|
||||
/**
|
||||
* String-keyed neighbor lookup — the internal implementation behind
|
||||
* {@link getNeighbors}. Kept UUID-based because the LSM trees are keyed by
|
||||
* UUID; only the public contract speaks BigInt.
|
||||
*/
|
||||
private async getNeighborUuids(
|
||||
id: string,
|
||||
options?: {
|
||||
direction?: 'in' | 'out' | 'both'
|
||||
limit?: number
|
||||
offset?: number
|
||||
}
|
||||
): Promise<string[]> {
|
||||
const startTime = performance.now()
|
||||
const direction = options.direction || 'both'
|
||||
const direction = options?.direction || 'both'
|
||||
const neighbors = new Set<string>()
|
||||
|
||||
// Query LSM-trees with bloom filter optimization
|
||||
|
|
@ -255,8 +355,8 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
|
||||
// Apply pagination if requested
|
||||
if (options?.limit !== undefined || options?.offset !== undefined) {
|
||||
const offset = options.offset || 0
|
||||
const limit = options.limit !== undefined ? options.limit : result.length
|
||||
const offset = options?.offset || 0
|
||||
const limit = options?.limit !== undefined ? options.limit : result.length
|
||||
result = result.slice(offset, offset + limit)
|
||||
}
|
||||
|
||||
|
|
@ -271,40 +371,34 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
}
|
||||
|
||||
/**
|
||||
* Get verb IDs by source - Billion-scale optimization for getVerbsBySource
|
||||
* @description Verb ints for all edges originating at `sourceInt` (BigInt
|
||||
* boundary). O(log n) LSM-tree lookup with bloom filter optimization;
|
||||
* filters out deleted verb IDs (tombstone deletion workaround); pagination
|
||||
* support for entities with many relationships. Unknown entity int → empty
|
||||
* result. Resolve returned ints back to verb-id strings with
|
||||
* {@link verbIntsToIds}.
|
||||
*
|
||||
* O(log n) LSM-tree lookup with bloom filter optimization
|
||||
* Filters out deleted verb IDs (tombstone deletion workaround)
|
||||
* Added pagination support for entities with many relationships
|
||||
*
|
||||
* @param sourceId Source entity ID
|
||||
* @param options Optional configuration
|
||||
* @param options.limit Maximum number of verb IDs to return (default: all)
|
||||
* @param options.offset Number of verb IDs to skip (default: 0)
|
||||
* @returns Array of verb IDs originating from this source (excluding deleted, paginated if requested)
|
||||
* @param sourceInt - Source entity int (from the shared idMapper).
|
||||
* @param options - Optional limit/offset pagination.
|
||||
* @returns Verb ints originating from this source (excluding deleted).
|
||||
*
|
||||
* @example
|
||||
* // Get all verb IDs (backward compatible)
|
||||
* const all = await graphIndex.getVerbIdsBySource(sourceId)
|
||||
*
|
||||
* @example
|
||||
* // Get first 50 verb IDs
|
||||
* const page1 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 50 })
|
||||
*
|
||||
* @example
|
||||
* // Paginate through verb IDs
|
||||
* const page1 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 100, offset: 0 })
|
||||
* const page2 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 100, offset: 100 })
|
||||
* const verbInts = await graphIndex.getVerbIdsBySource(42n, { limit: 50 })
|
||||
* const verbIds = await graphIndex.verbIntsToIds(verbInts)
|
||||
*/
|
||||
async getVerbIdsBySource(
|
||||
sourceId: string,
|
||||
sourceInt: bigint,
|
||||
options?: {
|
||||
limit?: number
|
||||
offset?: number
|
||||
}
|
||||
): Promise<string[]> {
|
||||
): Promise<bigint[]> {
|
||||
await this.ensureInitialized()
|
||||
|
||||
const mapper = this.requireEntityIdMapper()
|
||||
const sourceId = mapper.getUuid(Number(sourceInt))
|
||||
if (sourceId === undefined) return []
|
||||
|
||||
const startTime = performance.now()
|
||||
const verbIds = await this.lsmTreeVerbsBySource.get(sourceId)
|
||||
const elapsed = performance.now() - startTime
|
||||
|
|
@ -314,56 +408,37 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
prodLog.warn(`GraphAdjacencyIndex: Slow getVerbIdsBySource for ${sourceId}: ${elapsed.toFixed(2)}ms`)
|
||||
}
|
||||
|
||||
// Filter out deleted verb IDs (tombstone deletion workaround)
|
||||
// LSM-tree retains all IDs, but verbIdSet tracks deletions
|
||||
const allIds = verbIds || []
|
||||
let result = allIds.filter(id => this.verbIdSet.has(id))
|
||||
|
||||
// Apply pagination if requested
|
||||
if (options?.limit !== undefined || options?.offset !== undefined) {
|
||||
const offset = options.offset || 0
|
||||
const limit = options.limit !== undefined ? options.limit : result.length
|
||||
result = result.slice(offset, offset + limit)
|
||||
}
|
||||
|
||||
return result
|
||||
return this.verbIdsToPaginatedInts(verbIds || [], options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get verb IDs by target - Billion-scale optimization for getVerbsByTarget
|
||||
* @description Verb ints for all edges pointing at `targetInt` (BigInt
|
||||
* boundary). O(log n) LSM-tree lookup with bloom filter optimization;
|
||||
* filters out deleted verb IDs (tombstone deletion workaround); pagination
|
||||
* support for popular target entities. Unknown entity int → empty result.
|
||||
* Resolve returned ints back to verb-id strings with {@link verbIntsToIds}.
|
||||
*
|
||||
* O(log n) LSM-tree lookup with bloom filter optimization
|
||||
* Filters out deleted verb IDs (tombstone deletion workaround)
|
||||
* Added pagination support for popular target entities
|
||||
*
|
||||
* @param targetId Target entity ID
|
||||
* @param options Optional configuration
|
||||
* @param options.limit Maximum number of verb IDs to return (default: all)
|
||||
* @param options.offset Number of verb IDs to skip (default: 0)
|
||||
* @returns Array of verb IDs pointing to this target (excluding deleted, paginated if requested)
|
||||
* @param targetInt - Target entity int (from the shared idMapper).
|
||||
* @param options - Optional limit/offset pagination.
|
||||
* @returns Verb ints pointing to this target (excluding deleted).
|
||||
*
|
||||
* @example
|
||||
* // Get all verb IDs (backward compatible)
|
||||
* const all = await graphIndex.getVerbIdsByTarget(targetId)
|
||||
*
|
||||
* @example
|
||||
* // Get first 50 verb IDs
|
||||
* const page1 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 50 })
|
||||
*
|
||||
* @example
|
||||
* // Paginate through verb IDs
|
||||
* const page1 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 100, offset: 0 })
|
||||
* const page2 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 100, offset: 100 })
|
||||
* const verbInts = await graphIndex.getVerbIdsByTarget(42n, { limit: 50 })
|
||||
* const verbIds = await graphIndex.verbIntsToIds(verbInts)
|
||||
*/
|
||||
async getVerbIdsByTarget(
|
||||
targetId: string,
|
||||
targetInt: bigint,
|
||||
options?: {
|
||||
limit?: number
|
||||
offset?: number
|
||||
}
|
||||
): Promise<string[]> {
|
||||
): Promise<bigint[]> {
|
||||
await this.ensureInitialized()
|
||||
|
||||
const mapper = this.requireEntityIdMapper()
|
||||
const targetId = mapper.getUuid(Number(targetInt))
|
||||
if (targetId === undefined) return []
|
||||
|
||||
const startTime = performance.now()
|
||||
const verbIds = await this.lsmTreeVerbsByTarget.get(targetId)
|
||||
const elapsed = performance.now() - startTime
|
||||
|
|
@ -373,19 +448,42 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
prodLog.warn(`GraphAdjacencyIndex: Slow getVerbIdsByTarget for ${targetId}: ${elapsed.toFixed(2)}ms`)
|
||||
}
|
||||
|
||||
// Filter out deleted verb IDs (tombstone deletion workaround)
|
||||
// LSM-tree retains all IDs, but verbIdSet tracks deletions
|
||||
const allIds = verbIds || []
|
||||
return this.verbIdsToPaginatedInts(verbIds || [], options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared tail for the verb-int read methods: drop tombstoned ids (LSM trees
|
||||
* retain all ids; verbIdSet tracks deletions), apply pagination, and intern
|
||||
* the survivors to verb ints. Interning on the read path is safe — ids are
|
||||
* assigned deterministically within the process lifetime and never persisted.
|
||||
*/
|
||||
private verbIdsToPaginatedInts(
|
||||
allIds: string[],
|
||||
options?: { limit?: number; offset?: number }
|
||||
): bigint[] {
|
||||
let result = allIds.filter(id => this.verbIdSet.has(id))
|
||||
|
||||
// Apply pagination if requested
|
||||
if (options?.limit !== undefined || options?.offset !== undefined) {
|
||||
const offset = options.offset || 0
|
||||
const limit = options.limit !== undefined ? options.limit : result.length
|
||||
const offset = options?.offset || 0
|
||||
const limit = options?.limit !== undefined ? options.limit : result.length
|
||||
result = result.slice(offset, offset + limit)
|
||||
}
|
||||
|
||||
return result
|
||||
return result.map(id => BigInt(this.internVerbId(id)))
|
||||
}
|
||||
|
||||
/**
|
||||
* @description Batch reverse resolver: verb ints → verb-id strings (the
|
||||
* REQUIRED half of the 8.0 contract Brainy's warm cache feeds from). Reads
|
||||
* the in-process interning map populated by `addVerb`, `rebuild()`, and the
|
||||
* verb-id-set recovery path — the JS index derives the interning from
|
||||
* storage on cold start, so no sidecar persistence exists or is needed.
|
||||
* @param verbInts - Verb ints as returned by the verb-int read methods.
|
||||
* @returns One entry per input, order-preserving; `null` for unknown ints.
|
||||
*/
|
||||
async verbIntsToIds(verbInts: bigint[]): Promise<(string | null)[]> {
|
||||
return verbInts.map(verbInt => this.verbIntToId[Number(verbInt)] ?? null)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -529,15 +627,32 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
}
|
||||
|
||||
/**
|
||||
* Add relationship to index using LSM-tree storage
|
||||
* @description Add a relationship to the index (BigInt boundary). The
|
||||
* coordinator resolves both endpoint ints via `idMapper.getOrAssign` and
|
||||
* mirrors them onto `verb.sourceInt`/`verb.targetInt` before calling. The
|
||||
* JS index keys its LSM trees by the verb's endpoint UUIDs, so the int
|
||||
* params carry no extra information here — they exist for contract parity
|
||||
* with native providers whose trees are int-keyed.
|
||||
* @param verb - The verb to index (endpoint UUIDs are authoritative).
|
||||
* @param sourceInt - The source entity's interned int (contract parity).
|
||||
* @param targetInt - The target entity's interned int (contract parity).
|
||||
* @returns The interned verb int for `verb.id` (stable for the index lifetime).
|
||||
*/
|
||||
async addVerb(verb: GraphVerb): Promise<void> {
|
||||
async addVerb(verb: GraphVerb, sourceInt: bigint, targetInt: bigint): Promise<bigint> {
|
||||
await this.ensureInitialized()
|
||||
return BigInt(await this.indexVerb(verb))
|
||||
}
|
||||
|
||||
/**
|
||||
* String-keyed indexing core shared by {@link addVerb} and {@link rebuild}.
|
||||
* Returns the interned verb int.
|
||||
*/
|
||||
private async indexVerb(verb: GraphVerb): Promise<number> {
|
||||
const startTime = performance.now()
|
||||
|
||||
// Track verb ID (memory-efficient: IDs only, full objects loaded on-demand via UnifiedCache)
|
||||
this.verbIdSet.add(verb.id)
|
||||
const verbInt = this.internVerbId(verb.id)
|
||||
|
||||
// Add to LSM-trees (outgoing and incoming edges)
|
||||
await this.lsmTreeSource.add(verb.sourceId, verb.targetId)
|
||||
|
|
@ -561,12 +676,17 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
if (elapsed > 10.0) {
|
||||
prodLog.warn(`GraphAdjacencyIndex: Slow addVerb for ${verb.id}: ${elapsed.toFixed(2)}ms`)
|
||||
}
|
||||
|
||||
return verbInt
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove relationship from index
|
||||
* Note: LSM-tree edges persist (tombstone deletion not yet implemented)
|
||||
* Only removes from verb cache and updates counts
|
||||
* @description Remove a relationship from the index by its id string.
|
||||
* LSM-tree edges persist (tombstone deletion via verbIdSet filtering); the
|
||||
* verb's interned int is intentionally retained so previously returned verb
|
||||
* ints stay resolvable via {@link verbIntsToIds}.
|
||||
* @param verbId - The verb's UUID string.
|
||||
* @returns Resolves once the verb no longer appears in reads.
|
||||
*/
|
||||
async removeVerb(verbId: string): Promise<void> {
|
||||
await this.ensureInitialized()
|
||||
|
|
@ -624,6 +744,11 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
this.totalRelationshipsIndexed = 0
|
||||
// CRITICAL FIX - Clear relationship counts to prevent accumulation
|
||||
this.relationshipCountsByType.clear()
|
||||
// Re-derive verb-int interning from scratch — it's process-lifetime
|
||||
// derived state (never persisted), so a rebuild starts a fresh
|
||||
// generation of verb ints alongside the fresh verbIdSet.
|
||||
this.verbIdToInt.clear()
|
||||
this.verbIntToId = []
|
||||
|
||||
// Note: LSM-trees will be recreated from storage via their own initialization
|
||||
// Verb data will be loaded on-demand via UnifiedCache
|
||||
|
|
@ -655,7 +780,7 @@ export class GraphAdjacencyIndex implements GraphIndexProvider {
|
|||
confidence: verb.confidence,
|
||||
weight: verb.weight
|
||||
}
|
||||
await this.addVerb(graphVerb)
|
||||
await this.indexVerb(graphVerb)
|
||||
totalVerbs++
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue