David Snelling
9e307e457f
Brainy 7.30.0 introduced a memory-derived synchronous cap on `find({ limit })`
to prevent OOM. The cap was sound in intent but ~4x too conservative in
calibration: assumed 100 KB per result while typical entity footprint is 7-10 KB
(384-dim float32 vector ≈ 1.5 KB + standard fields + metadata). On a 900 MB
free-memory box the cap derived to 9000 — breaking common safety-cap patterns
like `find({ type, where, limit: 10_000 })` that typically return 10-500
entities. Surfaced as a runtime regression with cascading 500s degrading
production dashboards.
Three concurrent fixes:
A. RECALIBRATE THE FORMULA
- src/utils/paramValidation.ts:175,196,212 — the three memory-derived priorities
(reservedQueryMemory / containerMemory / freeMemory) all divided by
100 * 1024 * 1024 (100 KB per result, ~10-15x over conservative). Replaced
with a new MAX_LIMIT_KB_PER_RESULT = 25 constant that matches observed
entity size.
- Result: 4 GB container cap goes 10_000 → 40_000; 2 GB cap goes 5_000 →
20_000; 900 MB free-memory cap goes 9_000 → ~36_000. 100k hard ceiling
unchanged. `maxQueryLimit` / `reservedQueryMemory` constructor overrides
unchanged in behavior.
B. TWO-TIER ENFORCEMENT (warn-then-throw)
- Below cap (limit <= maxLimit): silent pass, unchanged.
- Soft tier (maxLimit < limit <= 2 * maxLimit): NEW — one-time warning per
call site (dedup keyed on caller stack frame + limit value), query
proceeds. Pre-7.30.2 code that relied on the cap silently allowing typical
safety-cap limits keeps working; the warning teaches the recipe so consumers
can fix it intentionally.
- Hard tier (limit > 2 * maxLimit): throw with the same teaching message
format. Real OOM territory; the cap stops being a recommendation and becomes
a guardrail.
- The 2x soft margin absorbs typical safety-cap patterns (limit: 10_000
against a 9 K-cap box) without disabling OOM protection. Real OOM territory
on a JS in-memory brain is hundreds of thousands of results, not 10x the
safety cap.
C. IMPROVED ERROR / WARNING MESSAGE
- Same shape as the 7.30.1 enforcement-error messages: state the problem,
name the three escape valves (maxQueryLimit / reservedQueryMemory /
pagination), include caller location, link to docs.
- Extracted findCallerLocation() helper from brainy.ts to a new
src/utils/callerLocation.ts so both the subtype enforcement (7.30.1) and
the limit enforcement (7.30.2) share one implementation without circular
imports.
DOCS
- New docs/guides/find-limits.md (public: true) — full reference: why the cap
exists, the four memory sources the auto-config considers, the three escape
valves with when-to-use-which guidance, and an explicit "pagination is the
future-proof pattern" callout (8.0 may tighten the cap further; pagination
keeps working unchanged).
- docs/api/README.md find() entry gets a one-paragraph `limit` tip + pointer
to the new guide.
- RELEASES.md v7.30.2 entry.
TESTS
- New tests/integration/find-limits.test.ts (9 tests): below-cap silent pass;
soft-tier warns once per call site (dedup verified by exercising same vs.
different source lines via wrapper closures); soft-tier message format
(names all three escape valves + docs link); soft-tier message includes
caller location; hard-tier throws; hard-tier message format same as
soft-tier; consumer maxQueryLimit override raises the cap and shifts both
tiers accordingly; pre-7.30.2 regression scenario explicitly covered.
- tests/unit/utils/memoryLimits.test.ts — 4 tests updated for the recalibrated
cap values (hardcoded expected numbers bumped 4x to match new 25 KB/result
assumption).
- tests/unit/utils/paramValidation.test.ts — auto-limit test extended to cover
the three-tier semantics (below-cap pass / soft-tier silent / hard-tier
throw).
- Existing suites unchanged: subtype-and-facets 26/26, verb-subtype-and-
enforcement 30/30, strict-mode-self-test 13/13. Unit 1468/1468.
CORTEX COMPATIBILITY
- Zero Cortex changes required. Every change is JS-side: formula recalibration
runs in ValidationConfig.constructor(), two-tier enforcement runs in
validateFindParams(), both fire before any storage / index / Cortex call.
- The new guide notes that Brainy 8.0's Datomic-style Db.find() may tighten
per-call limits to keep snapshot semantics cheap; pagination remains the
pattern that's guaranteed to keep working.
REPO-WIDE CLEANUP
Brainy is the only Soulcraft project that is open source. This commit also
scrubs closed-source product names and product-specific class/field references
from every tracked file in the repo (src/, docs/, tests/, RELEASES.md,
CHANGELOG.md). Consumer-reported bugs, regression scenarios, and release
notes now refer to "a consumer", "a downstream application", "a production
deployment", or "an internal report" — never to the named product. Two
product-named test files renamed to neutral diagnostic names. CLAUDE.md gains
a project-level guard rule documenting the policy and an example list of the
identifiers that may not appear in tracked code.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All four integration subtype + verb + strict + find-limits suites: 78/78
- npm run build: clean
- Closed-source product reference audit: clean
402 lines
12 KiB
TypeScript
402 lines
12 KiB
TypeScript
/**
|
||
* Unit tests for memory limit calculation and container detection (v5.11.0)
|
||
*
|
||
* Tests verify:
|
||
* - Container memory detection (cgroup v1/v2, env vars)
|
||
* - Smart memory limit calculation
|
||
* - Configuration overrides
|
||
* - Memory stats API
|
||
*/
|
||
|
||
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
||
import { Brainy } from '../../../src/brainy.js'
|
||
import { ValidationConfig } from '../../../src/utils/paramValidation.js'
|
||
import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'fs'
|
||
import { tmpdir } from 'os'
|
||
import { join } from 'path'
|
||
|
||
describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
||
let testDir: string
|
||
let originalEnv: Record<string, string | undefined>
|
||
|
||
beforeEach(() => {
|
||
testDir = mkdtempSync(join(tmpdir(), 'brainy-memory-test-'))
|
||
|
||
// Save original environment variables
|
||
originalEnv = {
|
||
CLOUD_RUN_MEMORY: process.env.CLOUD_RUN_MEMORY,
|
||
MEMORY_LIMIT: process.env.MEMORY_LIMIT
|
||
}
|
||
|
||
// Reset ValidationConfig singleton before each test
|
||
ValidationConfig.reset()
|
||
})
|
||
|
||
afterEach(() => {
|
||
rmSync(testDir, { recursive: true, force: true })
|
||
|
||
// Restore original environment
|
||
Object.keys(originalEnv).forEach(key => {
|
||
if (originalEnv[key] === undefined) {
|
||
delete process.env[key]
|
||
} else {
|
||
process.env[key] = originalEnv[key]
|
||
}
|
||
})
|
||
|
||
// Reset ValidationConfig after each test
|
||
ValidationConfig.reset()
|
||
})
|
||
|
||
describe('Container Memory Detection', () => {
|
||
it('should detect Cloud Run memory limit from env var', () => {
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
expect(config.detectedContainerLimit).toBe(4 * 1024 * 1024 * 1024)
|
||
expect(config.limitBasis).toBe('containerMemory')
|
||
|
||
// 4GB * 0.25 = 1GB query memory = 10k limit
|
||
expect(config.maxLimit).toBeGreaterThan(5000)
|
||
})
|
||
|
||
it('should detect Cloud Run memory limit in Mi units', () => {
|
||
process.env.CLOUD_RUN_MEMORY = '512Mi'
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
expect(config.detectedContainerLimit).toBe(512 * 1024 * 1024)
|
||
expect(config.limitBasis).toBe('containerMemory')
|
||
|
||
// 512MB * 0.25 = 128MB query memory
|
||
expect(config.maxLimit).toBeGreaterThan(0)
|
||
})
|
||
|
||
it('should detect generic MEMORY_LIMIT env var', () => {
|
||
process.env.MEMORY_LIMIT = String(2 * 1024 * 1024 * 1024) // 2GB
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
expect(config.detectedContainerLimit).toBe(2 * 1024 * 1024 * 1024)
|
||
expect(config.limitBasis).toBe('containerMemory')
|
||
})
|
||
|
||
it('should fall back to free memory if no container detected', () => {
|
||
// No environment variables set
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
expect(config.limitBasis).toBe('freeMemory')
|
||
expect(config.maxLimit).toBeGreaterThan(0)
|
||
})
|
||
})
|
||
|
||
describe('Smart Memory Limit Calculation', () => {
|
||
it('should allocate 25% of container memory for queries', () => {
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
// 4 GB × 0.25 = 1 GB for queries; at 25 KB / result (7.30.2 calibration)
|
||
// → 1 GB / 25 KB = ~40_960 → floor to 40 × 1000 = 40_000.
|
||
// Pre-7.30.2 used 100 KB / result and returned 10_000 here.
|
||
expect(config.maxLimit).toBe(40000)
|
||
})
|
||
|
||
it('should respect absolute maximum of 100k', () => {
|
||
// Simulate huge container
|
||
process.env.MEMORY_LIMIT = String(100 * 1024 * 1024 * 1024) // 100GB
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
// Should cap at 100,000 even with huge memory
|
||
expect(config.maxLimit).toBe(100000)
|
||
})
|
||
|
||
it('should handle small containers gracefully', () => {
|
||
process.env.CLOUD_RUN_MEMORY = '512Mi' // Use 512MB instead of 128MB for realistic test
|
||
|
||
const config = ValidationConfig.getInstance()
|
||
|
||
// 512MB * 0.25 = 128MB for queries
|
||
// 128MB / 100MB = 1.28 floor to 1 * 1000 = 1000 limit
|
||
expect(config.maxLimit).toBeGreaterThan(0)
|
||
expect(config.limitBasis).toBe('containerMemory')
|
||
})
|
||
})
|
||
|
||
describe('Configuration Overrides', () => {
|
||
it('should respect maxQueryLimit override', () => {
|
||
const config = ValidationConfig.getInstance({ maxQueryLimit: 50000 })
|
||
|
||
expect(config.maxLimit).toBe(50000)
|
||
expect(config.limitBasis).toBe('override')
|
||
})
|
||
|
||
it('should respect reservedQueryMemory override', () => {
|
||
// Reserve 1 GB for queries; at 25 KB / result (7.30.2) → 1 GB / 25 KB
|
||
// = ~40_960 → floor to 40 × 1000 = 40_000. Pre-7.30.2 used 100 KB /
|
||
// result and this returned 10_000.
|
||
const config = ValidationConfig.getInstance({
|
||
reservedQueryMemory: 1 * 1024 * 1024 * 1024
|
||
})
|
||
|
||
expect(config.maxLimit).toBe(40000)
|
||
expect(config.limitBasis).toBe('reservedMemory')
|
||
})
|
||
|
||
it('should prioritize maxQueryLimit over reservedQueryMemory', () => {
|
||
const config = ValidationConfig.getInstance({
|
||
maxQueryLimit: 25000,
|
||
reservedQueryMemory: 1 * 1024 * 1024 * 1024
|
||
})
|
||
|
||
expect(config.maxLimit).toBe(25000)
|
||
expect(config.limitBasis).toBe('override')
|
||
})
|
||
|
||
it('should cap explicit overrides at 100k for safety', () => {
|
||
const config = ValidationConfig.getInstance({
|
||
maxQueryLimit: 200000 // Try to set above max
|
||
})
|
||
|
||
expect(config.maxLimit).toBe(100000) // Capped
|
||
expect(config.limitBasis).toBe('override')
|
||
})
|
||
})
|
||
|
||
describe('ValidationConfig Reconfiguration', () => {
|
||
it('should reconfigure singleton with new options', () => {
|
||
const config1 = ValidationConfig.getInstance()
|
||
const originalLimit = config1.maxLimit
|
||
|
||
// Reconfigure
|
||
const config2 = ValidationConfig.reconfigure({ maxQueryLimit: 30000 })
|
||
|
||
expect(config2.maxLimit).toBe(30000)
|
||
expect(config2.limitBasis).toBe('override')
|
||
|
||
// Verify singleton updated
|
||
const config3 = ValidationConfig.getInstance()
|
||
expect(config3.maxLimit).toBe(30000)
|
||
})
|
||
|
||
it('should reset singleton', () => {
|
||
const config1 = ValidationConfig.getInstance({ maxQueryLimit: 10000 })
|
||
expect(config1.maxLimit).toBe(10000)
|
||
|
||
ValidationConfig.reset()
|
||
|
||
const config2 = ValidationConfig.getInstance()
|
||
// Should recalculate based on system memory
|
||
expect(config2.maxLimit).not.toBe(10000)
|
||
})
|
||
})
|
||
|
||
describe('Brain Integration', () => {
|
||
it('should configure memory limits via Brain constructor', async () => {
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
maxQueryLimit: 15000,
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
expect(stats.limits.maxQueryLimit).toBe(15000)
|
||
expect(stats.limits.basis).toBe('override')
|
||
expect(stats.config.maxQueryLimit).toBe(15000)
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should configure reserved memory via Brain constructor', async () => {
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
reservedQueryMemory: 500 * 1024 * 1024, // 500 MB
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
// 500 MB / 25 KB per result (7.30.2 calibration) = ~20_000.
|
||
// Pre-7.30.2 used 100 KB / result and this returned 5000.
|
||
expect(stats.limits.maxQueryLimit).toBe(20000)
|
||
expect(stats.limits.basis).toBe('reservedMemory')
|
||
expect(stats.config.reservedQueryMemory).toBe(500 * 1024 * 1024)
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should auto-detect container limits when no config provided', async () => {
|
||
process.env.CLOUD_RUN_MEMORY = '2Gi'
|
||
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
expect(stats.memory.containerLimit).toBe(2 * 1024 * 1024 * 1024)
|
||
expect(stats.limits.basis).toBe('containerMemory')
|
||
// 2 GB × 0.25 = 512 MB query budget; at 25 KB per result (7.30.2) →
|
||
// 512 MB / 25 KB = ~20_971 → floor to 20 × 1000 = 20_000. Pre-7.30.2
|
||
// used 100 KB per result and this returned 5_000.
|
||
expect(stats.limits.maxQueryLimit).toBe(20000)
|
||
|
||
await brain.close()
|
||
})
|
||
})
|
||
|
||
describe('getMemoryStats() API', () => {
|
||
it('should return complete memory statistics', async () => {
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
maxQueryLimit: 20000,
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
// Memory stats
|
||
expect(stats.memory).toHaveProperty('heapUsed')
|
||
expect(stats.memory).toHaveProperty('heapTotal')
|
||
expect(stats.memory).toHaveProperty('external')
|
||
expect(stats.memory).toHaveProperty('rss')
|
||
expect(stats.memory).toHaveProperty('free')
|
||
expect(stats.memory).toHaveProperty('total')
|
||
expect(stats.memory).toHaveProperty('containerLimit')
|
||
|
||
expect(stats.memory.containerLimit).toBe(4 * 1024 * 1024 * 1024)
|
||
|
||
// Limits
|
||
expect(stats.limits.maxQueryLimit).toBe(20000)
|
||
expect(stats.limits.basis).toBe('override')
|
||
expect(stats.limits.maxQueryLength).toBeGreaterThan(0)
|
||
expect(stats.limits.maxVectorDimensions).toBe(384)
|
||
|
||
// Config
|
||
expect(stats.config.maxQueryLimit).toBe(20000)
|
||
|
||
// Recommendations
|
||
expect(Array.isArray(stats.recommendations)).toBe(true)
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should provide recommendations when appropriate', async () => {
|
||
// Large container but using free memory basis
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
// Don't set overrides, let it use containerMemory
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
expect(stats.recommendations).toBeDefined()
|
||
expect(stats.recommendations!.length).toBeGreaterThanOrEqual(0)
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should handle browser environment gracefully', async () => {
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
// Should not crash in any environment
|
||
expect(stats).toBeDefined()
|
||
expect(stats.limits).toBeDefined()
|
||
|
||
await brain.close()
|
||
})
|
||
})
|
||
|
||
describe('Production Scenarios', () => {
|
||
it('should handle 4GB Cloud Run container optimally', async () => {
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
// Allocates 1 GB (25% of 4 GB) for queries; at 25 KB per result
|
||
// (7.30.2 calibration) → 1 GB / 25 KB = ~40_960 → floor to 40 × 1000 =
|
||
// 40_000. Pre-7.30.2 used 100 KB per result and this returned 10_000.
|
||
expect(stats.memory.containerLimit).toBe(4 * 1024 * 1024 * 1024)
|
||
expect(stats.limits.maxQueryLimit).toBe(40000)
|
||
expect(stats.limits.basis).toBe('containerMemory')
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should allow manual override for power users', async () => {
|
||
process.env.CLOUD_RUN_MEMORY = '4Gi'
|
||
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
maxQueryLimit: 50000, // Power user wants higher limit
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
expect(stats.limits.maxQueryLimit).toBe(50000)
|
||
expect(stats.limits.basis).toBe('override')
|
||
|
||
// Should note override in recommendations
|
||
const overrideNote = stats.recommendations?.find(r => r.includes('override'))
|
||
expect(overrideNote).toBeDefined()
|
||
|
||
await brain.close()
|
||
})
|
||
|
||
it('should handle bare metal deployment (no container)', async () => {
|
||
// No container env vars
|
||
|
||
const brain = new Brainy({
|
||
storage: { type: 'memory' },
|
||
silent: true
|
||
})
|
||
|
||
await brain.init()
|
||
|
||
const stats = brain.getMemoryStats()
|
||
|
||
expect(stats.memory.containerLimit).toBeNull()
|
||
expect(stats.limits.basis).toBe('freeMemory')
|
||
expect(stats.limits.maxQueryLimit).toBeGreaterThan(0)
|
||
|
||
await brain.close()
|
||
})
|
||
})
|
||
})
|