open-brainy/src/index.ts
David Snelling 5b2cbf74e5 fix: warm() metadata surface routes through the active provider (warm hook added to the metadata contract); add maintenanceDebt() observability surface
A production deployment's warm report showed metadata: 'unavailable' under a
native metadata provider. brain.warm()'s metadata leg only duck-typed the
built-in JS manager's hydrateAll() method, which a native provider has no
reason to implement.

- MetadataIndexProvider (src/plugin.ts) gains an optional warm?(): Promise<void>
  hook, mirroring the existing vector and graph provider hooks. brain.warm()
  now checks the active provider's own warm() FIRST, falls back to the JS
  manager's hydrateAll() when absent, and reports 'unavailable' only when
  neither exists -- never init() as a stand-in, since a native provider's
  init() may be a cheap verify rather than a real warm.
- Tests (tests/unit/brainy/warm.test.ts): a live provider instance shaped to
  have warm() reports 'warmed' and the hook called with no hydrateAll
  fallback; shaped to have neither hook reports 'unavailable' (pins the
  honest branch); the unmodified built-in JS manager still reports 'warmed'
  via hydrateAll(), unchanged.

Additive scope agreed mid-flight with the native-provider team: a
maintenance-debt observability seam so an operator sees a grind coming
instead of discovering it as a CPU storm.

- New optional maintenanceDebt?(): Promise<ProviderMaintenanceDebt> hook on
  all three provider contracts (vector, metadata, graph -- the same three
  warm?() lives on). ProviderMaintenanceDebt is fields-all-optional: a
  provider reports only what it truly measures (pendingBytes, pendingItems,
  lastPassCompletedAt, lastPassOutcome, converging), never an estimate
  dressed as fact.
- New public brain.maintenanceDebt(): a pure passthrough -- for each surface
  it calls only the active provider's own hook and reports the payload
  verbatim, or 'unavailable' when absent. No thresholds, no polling, no
  JS-side estimation; the provider owns the numbers, the operator owns the
  policy.
- ProviderMaintenanceDebt, MaintenanceDebtReport, and MaintenanceDebtOutcome
  are exported from the package root.
- Tests (tests/unit/brainy/maintenance-debt.test.ts): hook present reports
  'reported' with the exact payload passed through; hook absent reports
  'unavailable' on every surface; mixed surfaces resolve independently of
  each other.

RELEASES.md gains the 8.10.1 entry covering both fixes above and this
feature, including the no-hot-retry contract from the prior commit.
2026-07-24 16:02:01 -07:00

598 lines
14 KiB
TypeScript

/**
* Brainy 8.0 - Your AI-Powered Second Brain
* 🧠⚛️ A multi-dimensional database with vector, graph, and relational storage
*
* Core Components:
* - Brainy: The unified database with Triple Intelligence
* - Triple Intelligence: Seamless fusion of vector + graph + field search
* - Db: Immutable, generation-pinned database values (now/transact/asOf/with)
* - Plugins: Extensible plugin system (cor, storage adapters)
* - Neural Import: AI-powered entity extraction & smart data import
*/
// Export main Brainy class
import { Brainy } from './brainy.js'
export { Brainy }
// The in-process change feed (brain.onChange) — event + listener types.
export type {
BrainyChangeEvent,
ChangeEventEntity,
ChangeEventRelation,
ChangeListener
} from './events/changeFeed.js'
// Temporal VFS — a file version entry (vfs.history / readFile({ asOf })).
export type { FileVersion } from './vfs/types.js'
// Export diagnostics result type
export type { DiagnosticsResult } from './brainy.js'
// brain.warm() — eager index/storage readiness report (per-surface honest
// outcome + timing). See the WarmReport JSDoc in brainy.ts.
export type { WarmReport, WarmOutcome } from './brainy.js'
// brain.maintenanceDebt() — per-surface passthrough of each active
// provider's self-reported background maintenance debt. See the
// MaintenanceDebtReport JSDoc in brainy.ts and ProviderMaintenanceDebt in
// plugin.ts for the measure-only-what-you-track contract.
export type { MaintenanceDebtReport, MaintenanceDebtOutcome } from './brainy.js'
export type {
GraphAuditReport,
GraphAuditDiscrepancy
} from './graph/graphAudit.js'
export {
checkOsLimits,
NOFILE_POOL_FLOOR,
MAX_MAP_COUNT_POOL_FLOOR
} from './utils/osLimits.js'
export type { OsLimitsReport } from './utils/osLimits.js'
// Export Brainy configuration and types
export type {
BrainyConfig,
Entity,
Relation,
Result,
AddParams,
UpdateParams,
RelateParams,
UpdateRelationParams,
FindParams,
SimilarParams,
GetOptions,
RelatedParams,
AddManyParams,
UpdateManyParams,
RemoveManyParams,
RelateManyParams,
BatchResult,
SubtypeRegistry,
FillSubtypeRule,
FillSubtypeRules,
FillSubtypesResult,
AggregateDefinition,
AggregateMetricDef,
AggregateSource,
AggregateQueryParams,
AggregateResult,
AggregateGroupState,
MetricState,
AggregationOp,
TimeWindowGranularity,
GroupByDimension,
AggregationProvider
} from './types/brainy.types.js'
// Reserved-field contract — the canonical list of Brainy-owned field names
// that may never appear inside a `metadata` bag (see docs/concepts/consistency-model.md)
export {
RESERVED_ENTITY_FIELDS,
RESERVED_RELATION_FIELDS,
splitNounMetadataRecord,
splitVerbMetadataRecord
} from './types/reservedFields.js'
export type {
ReservedEntityField,
ReservedRelationField,
EntityMetadataInput,
EntityMetadataPatch,
RelationMetadataInput,
RelationMetadataPatch,
NoReservedEntityKeys,
NoReservedRelationKeys,
SplitMetadataRecord
} from './types/reservedFields.js'
// Export Aggregation Engine
export { AggregationIndex, AggregateMaterializer, bucketTimestamp, parseBucketRange } from './aggregation/index.js'
// Export Neural Import (AI data understanding)
export { NeuralImport } from './neural/neuralImport.js'
export type {
NeuralAnalysisResult,
DetectedEntity,
DetectedRelationship,
NeuralInsight,
NeuralImportOptions
} from './neural/neuralImport.js'
// Export Neural Entity Extraction
export { NeuralEntityExtractor } from './neural/entityExtractor.js'
export { SmartExtractor } from './neural/SmartExtractor.js'
export { SmartRelationshipExtractor } from './neural/SmartRelationshipExtractor.js'
export type {
ExtractedEntity
} from './neural/entityExtractor.js'
export type {
ExtractionResult,
SmartExtractorOptions,
FormatContext
} from './neural/SmartExtractor.js'
export type {
RelationshipExtractionResult,
SmartRelationshipExtractorOptions
} from './neural/SmartRelationshipExtractor.js'
// Export distance functions for convenience
import {
euclideanDistance,
cosineDistance,
manhattanDistance,
dotProductDistance
} from './utils/index.js'
export {
euclideanDistance,
cosineDistance,
manhattanDistance,
dotProductDistance
}
// Export version utilities
export { getBrainyVersion } from './utils/version.js'
// Export plugin system
export type { BrainyPlugin, BrainyPluginContext, StorageAdapterFactory } from './plugin.js'
export { PluginRegistry } from './plugin.js'
// Export migration system
export { MigrationRunner, MIGRATIONS } from './migration/index.js'
export type { Migration, MigrationState, MigrationPreview, MigrationResult, MigrateOptions, MigrationError } from './migration/index.js'
// Export optimistic-concurrency types (7.31.0)
export { RevisionConflictError } from './transaction/RevisionConflictError.js'
// Named not-found errors — thrown by update/relate/updateRelation/similar/
// transact/with() when a referenced entity or relation does not exist.
export { EntityNotFoundError, RelationNotFoundError } from './errors/notFound.js'
// Base error + typed migration-lock error — thrown by any data-plane call while a
// brain runs its one-time 7.x→8.0 upgrade; catch to answer HTTP 503 + Retry-After.
export { BrainyError, MigrationInProgressError, GraphIndexNotReadyError, MetadataIndexNotReadyError, VectorIndexNotReadyError, ProtectedArtifactError, DerivedArtifactMissingError } from './errors/brainyError.js'
export type { BrainyErrorType } from './errors/brainyError.js'
// ============= 8.0 Db API — generational MVCC =============
// Immutable database values: brain.now() / brain.transact() / brain.asOf() /
// db.with() / db.persist() / Brainy.load(). See src/db/ for the record layer.
export { Db } from './db/db.js'
// Portable graph export/import — db.export() / brain.export() / brain.import()
// (PortableGraph v1; identical wire format to the 7.x line).
export { isPortableGraph, validatePortableGraph } from './db/portableGraph.js'
export type {
PortableGraph,
PortableGraphEntity,
PortableGraphRelation,
ExportSelector,
ExportOptions,
ImportOptions,
ImportResult,
PortableGraphValidation
} from './db/portableGraph.js'
export {
GenerationConflictError,
SpeculativeOverlayError,
GenerationCompactedError,
StoreInconsistentError,
PendingFlushDurabilityError
} from './db/errors.js'
export type { UnreconciledRecord } from './db/errors.js'
export type {
TxOperation,
TxAddOperation,
TxUpdateOperation,
TxRemoveOperation,
TxRelateOperation,
TxUnrelateOperation,
TransactOptions,
TransactReceipt,
TxLogEntry,
CompactHistoryOptions,
CompactHistoryResult,
HistoryStats,
ChangedIds,
DiffResult,
HistoryVersion,
EntityHistory
} from './db/types.js'
// The generation fact log — sequential after-image scan surface
// (brain.scanFacts / brain.factSegmentPaths) for index heals and replays.
export type {
CommitFact,
FactOp,
FactScanBatch,
FactScanHandle
} from './db/factLog.js'
// The generalized family stamp — which source generation a projection
// reflects + the surface that verifies it whole; one verifier, both member
// modes (enumerated byte-exact / rollup invariants).
export { readFamilyStamp, verifyFamilyStamp, ENTITY_TREE_STAMP_PATH } from './db/familyStamp.js'
export type { FamilyStamp, StampMembers, StampVerdict } from './db/familyStamp.js'
// Optional provider capability for generation-aware native indexes
export { isVersionedIndexProvider } from './plugin.js'
export type { VersionedIndexProvider } from './plugin.js'
export type { ProviderInvariantReport, InvariantResult, InvariantHeal } from './plugin.js'
// Optional provider self-report of outstanding background maintenance work
// (compaction, deferred writes, etc.) — the payload type for
// brain.maintenanceDebt(). See the measure-only-what-you-track contract on
// ProviderMaintenanceDebt in plugin.ts.
export type { ProviderMaintenanceDebt } from './plugin.js'
// Optional native graph-acceleration engine (cor 3.0) — the published provider
// contract + its columnar wire types. Brainy feature-detects an implementation
// and falls back to its pure-TS adjacency when absent.
export type {
GraphAccelerationProvider,
Subgraph,
OpaqueIdSet,
GraphTraversalDirection,
TraverseOptions,
EdgesForNodeOptions,
GraphCursorHandle,
GraphCursorOptions,
GraphCursorChunk,
GraphScores,
GraphCommunities,
GraphPath,
RankOptions,
CommunitiesOptions,
PathOptions,
SampleOptions,
MostConnectedOptions
} from './plugin.js'
// Export embedding functionality
import {
UniversalSentenceEncoder,
TransformerEmbedding,
createEmbeddingFunction,
defaultEmbeddingFunction,
batchEmbed,
embeddingFunctions
} from './utils/embedding.js'
// Export logging utilities
import {
logger,
LogLevel,
configureLogger,
createModuleLogger
} from './utils/logger.js'
// Chat system removed - was returning fake responses
// Export performance and optimization utilities
import {
getGlobalBackpressure,
AdaptiveBackpressure
} from './utils/adaptiveBackpressure.js'
import {
getGlobalPerformanceMonitor,
PerformanceMonitor
} from './utils/performanceMonitor.js'
// Export environment utilities
import { isNode } from './utils/environment.js'
export {
UniversalSentenceEncoder,
TransformerEmbedding,
createEmbeddingFunction,
defaultEmbeddingFunction,
batchEmbed,
embeddingFunctions,
// Environment utilities
isNode,
// Logging utilities
logger,
LogLevel,
configureLogger,
createModuleLogger,
// Performance and optimization utilities
getGlobalBackpressure,
AdaptiveBackpressure,
getGlobalPerformanceMonitor,
PerformanceMonitor
}
// Export storage adapters (Brainy 8.0 — filesystem + memory only).
import { MemoryStorage, createStorage } from './storage/storageFactory.js'
export { MemoryStorage, createStorage }
// FileSystemStorage is exported separately to avoid browser build issues.
export { FileSystemStorage } from './storage/adapters/fileSystemStorage.js'
// Export types
import type {
Vector,
VectorDocument,
SearchResult,
DistanceFunction,
EmbeddingFunction,
EmbeddingModel,
HNSWNoun,
HNSWVerb,
HNSWConfig,
StorageAdapter,
DerivedFamilyDeclaration
} from './coreTypes.js'
// Export vector index implementation (the JS HNSW path)
import { JsHnswVectorIndex } from './hnsw/hnswIndex.js'
export { JsHnswVectorIndex }
export type {
Vector,
VectorDocument,
SearchResult,
DistanceFunction,
EmbeddingFunction,
EmbeddingModel,
HNSWNoun,
HNSWVerb,
HNSWConfig,
StorageAdapter,
DerivedFamilyDeclaration
}
// Export graph types
import type {
GraphNoun,
GraphVerb,
EmbeddedGraphVerb,
Person,
Organization,
Location,
Thing,
Concept,
Event,
Agent,
Organism,
Substance,
Quality,
TimeInterval,
Function,
Proposition,
Document,
Media,
File,
Message,
Collection,
Dataset,
Product,
Service,
Task,
Project,
Process,
State,
Role,
Language,
Currency,
Measurement,
Hypothesis,
Experiment,
Contract,
Regulation,
Interface,
Resource,
Custom,
SocialGroup,
Institution,
Norm,
InformationContent,
InformationBearer,
Relationship
} from './types/graphTypes.js'
import { NounType, VerbType } from './types/graphTypes.js'
export type {
GraphNoun,
GraphVerb,
EmbeddedGraphVerb,
Person,
Organization,
Location,
Thing,
Concept,
Event,
Agent,
Organism,
Substance,
Quality,
TimeInterval,
Function,
Proposition,
Document,
Media,
File,
Message,
Collection,
Dataset,
Product,
Service,
Task,
Project,
Process,
State,
Role,
Language,
Currency,
Measurement,
Hypothesis,
Experiment,
Contract,
Regulation,
Interface,
Resource,
Custom,
SocialGroup,
Institution,
Norm,
InformationContent,
InformationBearer,
Relationship
}
// Export type utility functions
import { getNounTypes, getVerbTypes, getNounTypeMap, getVerbTypeMap } from './utils/typeUtils.js'
// Export BrainyTypes for type validation and lookup
import { BrainyTypes } from './utils/brainyTypes.js'
export {
NounType,
VerbType,
getNounTypes,
getVerbTypes,
getNounTypeMap,
getVerbTypeMap,
// BrainyTypes - type validation and lookup
BrainyTypes
}
// Export MCP (Model Control Protocol) components
import {
BrainyMCPAdapter,
BrainyMCPService
} from './mcp/index.js' // Import from mcp/index.js
import {
MCPRequest,
MCPResponse,
MCPDataAccessRequest,
MCPToolExecutionRequest,
MCPSystemInfoRequest,
MCPAuthenticationRequest,
MCPRequestType,
MCPServiceOptions,
MCPTool,
MCP_VERSION
} from './types/mcpTypes.js'
export {
// MCP classes
BrainyMCPAdapter,
BrainyMCPService,
// MCP types
MCPRequestType,
MCP_VERSION
}
export type {
MCPRequest,
MCPResponse,
MCPDataAccessRequest,
MCPToolExecutionRequest,
MCPSystemInfoRequest,
MCPAuthenticationRequest,
MCPServiceOptions,
MCPTool
}
// ============= Integration Hub =============
// Connect Brainy to Excel, Power BI, Google Sheets, and more
// Enable with: new Brainy({ integrations: true })
// Hub class (used internally by brain.hub, also available for advanced use)
export {
IntegrationHub,
createIntegrationHub
} from './integrations/index.js'
export type {
IntegrationHubConfig,
IntegrationRequest,
IntegrationResponse
} from './integrations/index.js'
// Re-export IntegrationsConfig from types (for TypeScript users)
export type { IntegrationsConfig } from './types/brainy.types.js'
// Core infrastructure
export {
EventBus,
TabularExporter,
IntegrationBase,
IntegrationLoader,
createIntegrationLoader,
detectEnvironment,
INTEGRATION_CATALOG
} from './integrations/index.js'
// Integration types
export type {
BrainyEvent,
EventFilter,
EventHandler,
EventSubscription,
TabularRow,
RelationTabularRow,
TabularExporterConfig,
IntegrationConfig,
IntegrationHealthStatus,
HTTPIntegration,
StreamingIntegration,
IntegrationType,
RuntimeEnvironment,
IntegrationInfo,
IntegrationLoaderConfig,
ODataQueryOptions,
WebhookRegistration,
WebhookDeliveryResult
} from './integrations/index.js'
// Concrete integrations
export {
GoogleSheetsIntegration,
ODataIntegration,
SSEIntegration,
WebhookIntegration
} from './integrations/index.js'
export type {
GoogleSheetsConfig,
ODataConfig,
SSEConfig,
WebhookConfig
} from './integrations/index.js'
// OData utilities (advanced)
export {
parseODataQuery,
parseFilter,
parseOrderBy,
parseSelect,
odataToFindParams,
applyFilter,
applySelect,
applyOrderBy,
applyPagination,
generateEdmx,
generateMetadataJson,
generateServiceDocument
} from './integrations/index.js'