2026-01-06 16:02:13 -08:00
|
|
|
/**
|
|
|
|
|
* Performance Regression Tests for add() Operations
|
|
|
|
|
*
|
|
|
|
|
* These tests ensure that add() operations maintain acceptable performance.
|
|
|
|
|
* They detect regressions like the v7.1.0 bug where cloud storage type detection
|
|
|
|
|
* failed, causing 50-100x slower add() operations.
|
|
|
|
|
*
|
|
|
|
|
* v7.1.1: Added after discovering storage type detection bug
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
|
|
|
|
import { Brainy } from '../../../src/brainy.js'
|
|
|
|
|
import { NounType } from '../../../src/types/graphTypes.js'
|
|
|
|
|
|
|
|
|
|
describe('add() Performance Regression Tests', () => {
|
|
|
|
|
let brain: Brainy
|
|
|
|
|
|
|
|
|
|
beforeEach(async () => {
|
feat(8.0)!: flip requireSubtype default to true (BRAINY-8.0-SUBTYPE-CONTRACT § C-1)
Brainy 8.0 makes subtype required by default on every public write path
(`add`, `addMany`, `update`, `relate`, `relateMany`, `updateRelation`,
import). Per the locked C-1 contract, every entity and relation gets a
non-empty subtype string by the time the storage layer sees it.
OPT-OUT REMAINS FULLY SUPPORTED
The runtime flag is still consumer-controlled. Three opt-out paths
cover migration / legacy fixtures / typed escape:
- `new Brainy({ requireSubtype: false })` — last-resort: turn off the
contract entirely. Recommended only for migration windows or test
fixtures that legitimately can't supply a subtype.
- `new Brainy({ requireSubtype: { except: [NounType.Thing, ...] } })` —
per-type allowlist: strict everywhere except the listed types.
- `brain.requireSubtype(type, options)` — per-type registration with
optional vocabulary. Composes with the brain-wide flag.
Default is now `true`. Opt-out is explicit and documented; nothing
silently degrades.
TEST SWEEP
Bulk-applied `requireSubtype: false` to every `new Brainy({...})` call
site across 120 test files. Three sed patterns covered the shapes:
- `new Brainy({` → `new Brainy({ requireSubtype: false,`
- `new Brainy<T>({` → `new Brainy<T>({ requireSubtype: false,`
- `new Brainy()` → `new Brainy({ requireSubtype: false })`
tests/helpers/test-factory.ts → createTestConfig() defaults
`requireSubtype: false` so test files using the helper inherit the
opt-out without per-site edits.
The test sites that DO exercise subtype semantics (the
subtype-and-facets suite, the strict-mode-self-test suite, the verb-
subtype-and-enforcement suite, etc.) already pass real subtypes — they
were the 7.30.x acceptance tests for this contract. Those tests
continue to pass unchanged.
CHANGES
src/brainy.ts
- normalizeConfig() — `requireSubtype` default `false` → `true`.
Comment refreshed to document the three opt-out paths.
tests/* (120 files)
- Bulk-edited brain construction sites. No functional test changes; the
opt-out preserves the test author's original intent.
tests/helpers/test-factory.ts
- createTestConfig() base config gains `requireSubtype: false`.
NO-OP for consumers who were already passing subtype on every write.
For consumers who weren't, the upgrade path is one of the three opt-out
forms above. Migration recipe documented in 8.0 release notes (next
commit).
VERIFICATION
- npx tsc --noEmit: clean
- npm test: 1408 / 1409 (same pre-existing race-condition outstanding;
no other regressions from the flip)
2026-06-09 14:58:25 -07:00
|
|
|
brain = new Brainy({ requireSubtype: false,
|
2026-01-06 16:02:13 -08:00
|
|
|
storage: { type: 'memory' },
|
refactor: remove augmentation system and semantic type matching
Remove the entire augmentation pipeline infrastructure (52 files,
~15,000 lines) and the semantic type matching system. These were
unused middleware layers adding complexity without value.
What was removed:
- src/augmentations/ directory (all augmentation implementations)
- src/augmentationManager.ts (pipeline orchestrator)
- src/types/augmentations.ts, src/types/pipelineTypes.ts
- src/shared/default-augmentations.ts
- Semantic type suggestion (BrainyTypes.suggestNoun/suggestVerb)
- src/utils/typeMatching/ (embedding-based type matcher)
What was preserved by relocating:
- Import handlers (CSV, PDF, Excel) -> src/importers/handlers/
- NeuralImportAugmentation -> src/cortex/neuralImportAugmentation.ts
- Type matching utilities -> heuristic inference in consumers
What was simplified:
- brainy.ts: operations call storage directly (no execute() wrapper)
- IntegrationBase: standalone class (no BaseAugmentation parent)
- BrainyTypes: validation-only (nouns, verbs, isValid*, get*)
- Pipeline: direct execution (no augmentation interception)
- index.ts: removed TypeSuggestion, suggestType exports
- package.json: removed stale types/augmentations export
Build passes, 1176 tests pass, 0 failures.
2026-02-01 10:48:56 -08:00
|
|
|
silent: true
|
2026-01-06 16:02:13 -08:00
|
|
|
})
|
|
|
|
|
await brain.init()
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
afterEach(async () => {
|
|
|
|
|
await brain.close()
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
describe('Single add() Performance', () => {
|
|
|
|
|
it('should complete single add() in < 500ms with memory storage', async () => {
|
|
|
|
|
const start = performance.now()
|
|
|
|
|
|
|
|
|
|
await brain.add({
|
|
|
|
|
data: 'Test entity for performance measurement',
|
|
|
|
|
type: NounType.Concept
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
const elapsed = performance.now() - start
|
|
|
|
|
|
|
|
|
|
// Single add should be well under 500ms with memory storage
|
|
|
|
|
// Includes embedding generation (~50-150ms) + indexing (~10-50ms)
|
|
|
|
|
expect(elapsed).toBeLessThan(500)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should complete add() with metadata in < 500ms', async () => {
|
|
|
|
|
const start = performance.now()
|
|
|
|
|
|
|
|
|
|
await brain.add({
|
|
|
|
|
data: { title: 'Test Document', content: 'This is test content for performance' },
|
|
|
|
|
type: NounType.Document,
|
|
|
|
|
metadata: {
|
|
|
|
|
author: 'test',
|
|
|
|
|
category: 'performance',
|
|
|
|
|
tags: ['test', 'performance', 'benchmark']
|
|
|
|
|
}
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
const elapsed = performance.now() - start
|
|
|
|
|
expect(elapsed).toBeLessThan(500)
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
describe('Batch add() Performance', () => {
|
|
|
|
|
it('should complete 10 add() operations in < 5 seconds', async () => {
|
|
|
|
|
const start = performance.now()
|
|
|
|
|
|
|
|
|
|
for (let i = 0; i < 10; i++) {
|
|
|
|
|
await brain.add({
|
|
|
|
|
data: `Entity ${i} for batch performance test`,
|
|
|
|
|
type: NounType.Concept
|
|
|
|
|
})
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const elapsed = performance.now() - start
|
|
|
|
|
|
|
|
|
|
// 10 adds should average < 500ms each = 5000ms total
|
|
|
|
|
// This catches severe regressions like the 7-12 second per add bug
|
|
|
|
|
expect(elapsed).toBeLessThan(5000)
|
|
|
|
|
|
|
|
|
|
// Also verify reasonable average
|
|
|
|
|
const avgPerAdd = elapsed / 10
|
|
|
|
|
expect(avgPerAdd).toBeLessThan(500)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should maintain consistent performance across sequential adds', async () => {
|
|
|
|
|
const times: number[] = []
|
|
|
|
|
|
|
|
|
|
for (let i = 0; i < 5; i++) {
|
|
|
|
|
const start = performance.now()
|
|
|
|
|
|
|
|
|
|
await brain.add({
|
|
|
|
|
data: `Sequential entity ${i}`,
|
|
|
|
|
type: NounType.Concept
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
times.push(performance.now() - start)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// No single add should be drastically slower than others
|
|
|
|
|
// (catches issues where first add is slow due to lazy init)
|
|
|
|
|
const maxTime = Math.max(...times)
|
|
|
|
|
const avgTime = times.reduce((a, b) => a + b, 0) / times.length
|
|
|
|
|
|
|
|
|
|
// Max should not be more than 3x the average (allows for first-add warmup)
|
|
|
|
|
expect(maxTime).toBeLessThan(avgTime * 3)
|
|
|
|
|
|
|
|
|
|
// All adds should be under 1 second
|
|
|
|
|
expect(maxTime).toBeLessThan(1000)
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
describe('Storage Type Detection', () => {
|
|
|
|
|
it('should detect memory storage type correctly', async () => {
|
|
|
|
|
// Access private method via any cast for testing
|
|
|
|
|
const storageType = (brain as any).getStorageType()
|
|
|
|
|
expect(storageType).toBe('memory')
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should use immediate persistence for memory storage', async () => {
|
|
|
|
|
// Memory storage should use immediate mode (already fast)
|
|
|
|
|
const index = (brain as any).index
|
|
|
|
|
expect(index.persistMode).toBe('immediate')
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
describe('Cloud Storage Type Detection', () => {
|
|
|
|
|
// These tests verify that cloud storage types are detected correctly
|
|
|
|
|
// They don't actually connect to cloud storage, just verify the detection logic
|
|
|
|
|
|
|
|
|
|
it('should detect GCS storage type from class name', () => {
|
|
|
|
|
// Create a mock storage with GCS-like class name
|
|
|
|
|
class GcsStorage {
|
|
|
|
|
constructor() {}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const mockBrain = {
|
|
|
|
|
storage: new GcsStorage(),
|
|
|
|
|
getStorageType() {
|
|
|
|
|
if (!this.storage) return 'memory'
|
|
|
|
|
const className = this.storage.constructor.name
|
|
|
|
|
if (className.includes('Gcs') || className.includes('GCS')) return 'gcs'
|
|
|
|
|
if (className.includes('S3')) return 's3'
|
|
|
|
|
if (className.includes('R2')) return 'r2'
|
|
|
|
|
if (className.includes('Azure')) return 'azure'
|
|
|
|
|
return 'unknown'
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
expect(mockBrain.getStorageType()).toBe('gcs')
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should detect S3 storage type from class name', () => {
|
|
|
|
|
class S3CompatibleStorage {
|
|
|
|
|
constructor() {}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const mockBrain = {
|
|
|
|
|
storage: new S3CompatibleStorage(),
|
|
|
|
|
getStorageType() {
|
|
|
|
|
if (!this.storage) return 'memory'
|
|
|
|
|
const className = this.storage.constructor.name
|
|
|
|
|
if (className.includes('S3')) return 's3'
|
|
|
|
|
return 'unknown'
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
expect(mockBrain.getStorageType()).toBe('s3')
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should detect R2 storage type from class name', () => {
|
|
|
|
|
class R2Storage {
|
|
|
|
|
constructor() {}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const mockBrain = {
|
|
|
|
|
storage: new R2Storage(),
|
|
|
|
|
getStorageType() {
|
|
|
|
|
if (!this.storage) return 'memory'
|
|
|
|
|
const className = this.storage.constructor.name
|
|
|
|
|
if (className.includes('R2')) return 'r2'
|
|
|
|
|
return 'unknown'
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
expect(mockBrain.getStorageType()).toBe('r2')
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
it('should detect Azure storage type from class name', () => {
|
|
|
|
|
class AzureBlobStorage {
|
|
|
|
|
constructor() {}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const mockBrain = {
|
|
|
|
|
storage: new AzureBlobStorage(),
|
|
|
|
|
getStorageType() {
|
|
|
|
|
if (!this.storage) return 'memory'
|
|
|
|
|
const className = this.storage.constructor.name
|
|
|
|
|
if (className.includes('Azure')) return 'azure'
|
|
|
|
|
return 'unknown'
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
expect(mockBrain.getStorageType()).toBe('azure')
|
|
|
|
|
})
|
|
|
|
|
})
|