test(8.0): Tier-1 integration via deterministic embedder (suite runnable again)

The integration suite loaded the real WASM model for every test, so a full run
took ~hours and reliably aborted (vitest reporter timeout) — which is why it was
never gated and silently rotted. Brainy already separates the embedder from all
other logic, so running integration with a deterministic embedder keeps every
code path real (HNSW build+search, graph traversal, metadata filtering,
transactions, storage/sharding, VFS, aggregation, import) while making the suite
complete in ~2 min on ANY machine — no GPU, no 16 GB requirement.

- Add isDeterministicEmbedMode() — a single source of truth for the test-embedder
  switch (BRAINY_DETERMINISTIC_EMBEDDINGS, plus the legacy BRAINY_UNIT_TEST alias),
  used by EmbeddingManager (init/embed/embedBatch) and brainy's eager-init guard.
- setup-integration.ts opts into it: Tier 1 = functional end-to-end. Real-model
  semantic quality + the real embedding pipeline move to a Tier-2 suite.

Full integration now runs 482 passing / 583 in ~2 min (was un-completable). The
remaining failures are pre-existing test rot, fixed next. Unit 1414 green.
This commit is contained in:
David Snelling 2026-06-16 12:44:14 -07:00
parent e31ba894f8
commit 542b52ede9
4 changed files with 84 additions and 67 deletions

View file

@ -1,60 +1,44 @@
/**
* Integration Test Setup - REAL AI functionality
*
* This setup enables real AI models for integration testing
* Requires high memory environment (16GB+ RAM)
* Integration Test Setup Tier 1: Functional end-to-end (deterministic embedder)
*
* These tests exercise every real code path HNSW build + search, graph
* traversal, metadata filtering, triple-intelligence composition, MVCC
* transactions, sharding/persistence, VFS, aggregation, import but with a
* DETERMINISTIC embedder instead of the real WASM transformer. The embedding's
* semantic *quality* is irrelevant to these assertions (which are structural:
* counts, ids, relationships, self-retrieval, transaction outcomes); making the
* vector deterministic turns a multi-hour, un-completable suite into a sub-minute
* one that any contributor can run on any machine, no GPU or 16 GB required.
*
* Semantic relevance + the real embedding pipeline are covered separately by the
* Tier-2 semantic suite (`tests/semantic/`, real model) see
* `tests/configs/vitest.semantic.config.ts`.
*/
beforeAll(async () => {
console.log('🤖 Integration Test Environment: Using REAL AI models')
console.log('⚠️ Requires 16GB+ RAM - this is normal for AI testing')
// Set up environment for real AI testing
process.env.BRAINY_INTEGRATION_TEST = 'true'
process.env.BRAINY_MODELS_PATH = './models'
process.env.BRAINY_ALLOW_REMOTE_MODELS = 'false' // Use local models only
// Set memory limits and optimizations
process.env.ORT_DISABLE_MEMORY_ARENA = '1'
process.env.ORT_DISABLE_MEMORY_PATTERN = '1'
process.env.ORT_INTRA_OP_NUM_THREADS = '2'
process.env.ORT_INTER_OP_NUM_THREADS = '2'
// Mark as integration test environment
;(globalThis as any).__BRAINY_INTEGRATION_TEST__ = true
// Check memory availability
const availableMemoryGB = process.env.NODE_OPTIONS?.includes('max-old-space-size')
? parseInt(process.env.NODE_OPTIONS.match(/--max-old-space-size=(\d+)/)?.[1] || '0') / 1024
: 4
console.log(`📊 Node.js heap limit: ${availableMemoryGB.toFixed(1)}GB`)
if (availableMemoryGB < 8) {
console.warn('⚠️ WARNING: Less than 8GB allocated for integration tests')
console.warn(' Recommended: NODE_OPTIONS="--max-old-space-size=16384"')
console.warn(' Tests may fail due to insufficient memory')
}
}, 60000) // 1 minute timeout for setup
beforeAll(() => {
console.log('🧪 Integration (Tier 1): deterministic embedder, all other code paths real')
afterAll(async () => {
// Clean up
delete process.env.BRAINY_INTEGRATION_TEST
delete (globalThis as any).__BRAINY_INTEGRATION_TEST__
// Force garbage collection if available
if (global.gc) {
global.gc()
}
}, 30000) // 30 second timeout for cleanup
// Deterministic embeddings — see src/embeddings/deterministicEmbedMode.ts.
process.env.BRAINY_DETERMINISTIC_EMBEDDINGS = 'true'
;(globalThis as { __BRAINY_DETERMINISTIC_EMBED__?: boolean }).__BRAINY_DETERMINISTIC_EMBED__ = true
// Utility function to skip tests if not enough memory
export function requiresMemory(minGB: number) {
const availableMemoryGB = process.env.NODE_OPTIONS?.includes('max-old-space-size')
? parseInt(process.env.NODE_OPTIONS.match(/--max-old-space-size=(\d+)/)?.[1] || '0') / 1024
: 4
if (availableMemoryGB < minGB) {
throw new Error(`Test requires ${minGB}GB memory, only ${availableMemoryGB.toFixed(1)}GB allocated`)
}
}
// Marker retained for any test that branches on the integration environment.
;(globalThis as { __BRAINY_INTEGRATION_TEST__?: boolean }).__BRAINY_INTEGRATION_TEST__ = true
})
afterAll(() => {
delete process.env.BRAINY_DETERMINISTIC_EMBEDDINGS
delete (globalThis as { __BRAINY_DETERMINISTIC_EMBED__?: boolean }).__BRAINY_DETERMINISTIC_EMBED__
delete (globalThis as { __BRAINY_INTEGRATION_TEST__?: boolean }).__BRAINY_INTEGRATION_TEST__
})
/**
* No-op retained for back-compat. Tier-1 integration tests use the deterministic
* embedder, so the real model's 16 GB requirement no longer applies there is
* nothing to gate on. (Real-model memory cost lives in the Tier-2 semantic suite.)
*
* @param _minGB - Ignored.
*/
export function requiresMemory(_minGB: number): void {
// Intentionally empty — see the doc comment above.
}