fix: recalibrate find({ limit }) cap + two-tier enforcement + caller location
Brainy 7.30.0 introduced a memory-derived synchronous cap on `find({ limit })`
to prevent OOM. The cap was sound in intent but ~4x too conservative in
calibration: assumed 100 KB per result while typical entity footprint is 7-10 KB
(384-dim float32 vector ≈ 1.5 KB + standard fields + metadata). On a 900 MB
free-memory box the cap derived to 9000 — breaking common safety-cap patterns
like `find({ type, where, limit: 10_000 })` that typically return 10-500
entities. Surfaced as a runtime regression with cascading 500s degrading
production dashboards.
Three concurrent fixes:
A. RECALIBRATE THE FORMULA
- src/utils/paramValidation.ts:175,196,212 — the three memory-derived priorities
(reservedQueryMemory / containerMemory / freeMemory) all divided by
100 * 1024 * 1024 (100 KB per result, ~10-15x over conservative). Replaced
with a new MAX_LIMIT_KB_PER_RESULT = 25 constant that matches observed
entity size.
- Result: 4 GB container cap goes 10_000 → 40_000; 2 GB cap goes 5_000 →
20_000; 900 MB free-memory cap goes 9_000 → ~36_000. 100k hard ceiling
unchanged. `maxQueryLimit` / `reservedQueryMemory` constructor overrides
unchanged in behavior.
B. TWO-TIER ENFORCEMENT (warn-then-throw)
- Below cap (limit <= maxLimit): silent pass, unchanged.
- Soft tier (maxLimit < limit <= 2 * maxLimit): NEW — one-time warning per
call site (dedup keyed on caller stack frame + limit value), query
proceeds. Pre-7.30.2 code that relied on the cap silently allowing typical
safety-cap limits keeps working; the warning teaches the recipe so consumers
can fix it intentionally.
- Hard tier (limit > 2 * maxLimit): throw with the same teaching message
format. Real OOM territory; the cap stops being a recommendation and becomes
a guardrail.
- The 2x soft margin absorbs typical safety-cap patterns (limit: 10_000
against a 9 K-cap box) without disabling OOM protection. Real OOM territory
on a JS in-memory brain is hundreds of thousands of results, not 10x the
safety cap.
C. IMPROVED ERROR / WARNING MESSAGE
- Same shape as the 7.30.1 enforcement-error messages: state the problem,
name the three escape valves (maxQueryLimit / reservedQueryMemory /
pagination), include caller location, link to docs.
- Extracted findCallerLocation() helper from brainy.ts to a new
src/utils/callerLocation.ts so both the subtype enforcement (7.30.1) and
the limit enforcement (7.30.2) share one implementation without circular
imports.
DOCS
- New docs/guides/find-limits.md (public: true) — full reference: why the cap
exists, the four memory sources the auto-config considers, the three escape
valves with when-to-use-which guidance, and an explicit "pagination is the
future-proof pattern" callout (8.0 may tighten the cap further; pagination
keeps working unchanged).
- docs/api/README.md find() entry gets a one-paragraph `limit` tip + pointer
to the new guide.
- RELEASES.md v7.30.2 entry.
TESTS
- New tests/integration/find-limits.test.ts (9 tests): below-cap silent pass;
soft-tier warns once per call site (dedup verified by exercising same vs.
different source lines via wrapper closures); soft-tier message format
(names all three escape valves + docs link); soft-tier message includes
caller location; hard-tier throws; hard-tier message format same as
soft-tier; consumer maxQueryLimit override raises the cap and shifts both
tiers accordingly; pre-7.30.2 regression scenario explicitly covered.
- tests/unit/utils/memoryLimits.test.ts — 4 tests updated for the recalibrated
cap values (hardcoded expected numbers bumped 4x to match new 25 KB/result
assumption).
- tests/unit/utils/paramValidation.test.ts — auto-limit test extended to cover
the three-tier semantics (below-cap pass / soft-tier silent / hard-tier
throw).
- Existing suites unchanged: subtype-and-facets 26/26, verb-subtype-and-
enforcement 30/30, strict-mode-self-test 13/13. Unit 1468/1468.
CORTEX COMPATIBILITY
- Zero Cortex changes required. Every change is JS-side: formula recalibration
runs in ValidationConfig.constructor(), two-tier enforcement runs in
validateFindParams(), both fire before any storage / index / Cortex call.
- The new guide notes that Brainy 8.0's Datomic-style Db.find() may tighten
per-call limits to keep snapshot semantics cheap; pagination remains the
pattern that's guaranteed to keep working.
REPO-WIDE CLEANUP
Brainy is the only Soulcraft project that is open source. This commit also
scrubs closed-source product names and product-specific class/field references
from every tracked file in the repo (src/, docs/, tests/, RELEASES.md,
CHANGELOG.md). Consumer-reported bugs, regression scenarios, and release
notes now refer to "a consumer", "a downstream application", "a production
deployment", or "an internal report" — never to the named product. Two
product-named test files renamed to neutral diagnostic names. CLAUDE.md gains
a project-level guard rule documenting the policy and an example list of the
identifiers that may not appear in tracked code.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All four integration subtype + verb + strict + find-limits suites: 78/78
- npm run build: clean
- Closed-source product reference audit: clean
This commit is contained in:
parent
34e8271c53
commit
9e307e457f
35 changed files with 819 additions and 154 deletions
38
CHANGELOG.md
38
CHANGELOG.md
|
|
@ -643,7 +643,7 @@ v7.0.0 introduced **breaking changes** to the embedding system:
|
|||
| copy 15 files | ~120s | ~20-40s | 3-6x faster |
|
||||
| move 15 files | ~240s | ~40-60s | 4-6x faster |
|
||||
|
||||
Requested by: Soulcraft Workshop team (BRAINY-VFS-RMDIR-PERFORMANCE)
|
||||
Requested by: a consumer team (BRAINY-VFS-RMDIR-PERFORMANCE)
|
||||
|
||||
### [6.3.2](https://github.com/soulcraftlabs/brainy/compare/v6.3.1...v6.3.2) (2025-12-09)
|
||||
|
||||
|
|
@ -686,7 +686,7 @@ Requested by: Soulcraft Workshop team (BRAINY-VFS-RMDIR-PERFORMANCE)
|
|||
**Fixed VFS tree operations on cloud storage (GCS, S3, Azure, R2, OPFS)**
|
||||
|
||||
**Issue:** Despite v6.1.0's PathResolver optimization, `vfs.getTreeStructure()` remained critically slow on cloud storage:
|
||||
- **Workshop Production (GCS):** 5,304ms for tree with maxDepth=2
|
||||
- **Production (GCS) deployment:** 5,304ms for tree with maxDepth=2
|
||||
- **Root Cause:** Tree traversal made 111+ separate storage calls (one per directory)
|
||||
- **Why v6.1.0 didn't help:** v6.1.0 optimized path→ID resolution, but tree traversal still called `getChildren()` 111+ times
|
||||
|
||||
|
|
@ -714,7 +714,7 @@ Result: 111 storage calls → 1 storage call
|
|||
- `src/vfs/VirtualFileSystem.ts:730-762` - Updated `getDescendants()` to use batch fetch
|
||||
|
||||
**Impact:**
|
||||
- ✅ Workshop file explorer now loads instantly on GCS
|
||||
- ✅ Consumer file explorer now loads instantly on GCS
|
||||
- ✅ Clean architecture: one code path, no fallbacks
|
||||
- ✅ Production-scale: uses in-memory graph + single batch fetch
|
||||
- ✅ Works for ALL storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem)
|
||||
|
|
@ -1053,7 +1053,7 @@ for (const id of pageIds) {
|
|||
**Fix:** Set `isInitialized = true` at the START of `BaseStorage.init()` (before any initialization work) to prevent recursive calls. Flag is reset to `false` on error to allow retries.
|
||||
|
||||
**Impact:**
|
||||
- ✅ Fixes production blocker reported by Workshop team
|
||||
- ✅ Fixes production blocker reported by a consumer team
|
||||
- ✅ All 8 storage adapters fixed (FileSystem, Memory, S3, R2, GCS, Azure, OPFS, Historical)
|
||||
- ✅ Init completes in ~1 second on fresh installation (was hanging indefinitely)
|
||||
- ✅ No new test failures introduced (1178 tests passing)
|
||||
|
|
@ -1333,7 +1333,7 @@ See comprehensive guides:
|
|||
v5.10.0 reintroduced a critical bug where `BlobStorage.read()` was hashing wrapped binary data instead of unwrapped content, causing all blob integrity checks to fail:
|
||||
- **Symptom**: `Blob integrity check failed: <hash>` errors on every VFS file read
|
||||
- **Root Cause**: Missing defense-in-depth unwrap verification in `BlobStorage.read()`
|
||||
- **Impact**: 100% failure rate for VFS file operations in Workshop application
|
||||
- **Impact**: 100% failure rate for VFS file operations in A consumer application
|
||||
|
||||
### The Fix (v5.10.1)
|
||||
1. **Defense-in-Depth Unwrapping**: Added unwrap verification in `BlobStorage.read()` before hash check
|
||||
|
|
@ -1481,7 +1481,7 @@ Reverted storage internals to v5.6.3 implementation:
|
|||
- ✅ No 12+ second delays per entity
|
||||
|
||||
### Verification
|
||||
Workshop team (production users) should upgrade immediately:
|
||||
a consumer team (production users) should upgrade immediately:
|
||||
```bash
|
||||
npm install @soulcraft/brainy@5.7.1
|
||||
```
|
||||
|
|
@ -1616,10 +1616,10 @@ Expected behavior after upgrade:
|
|||
- Impact: All relationship queries via `getRelations()`, `getRelationships()`
|
||||
- Reference: `src/storage/baseStorage.ts:2030-2040`, `src/storage/baseStorage.ts:2081-2091`
|
||||
|
||||
* **Workshop blob integrity**: Verified v5.4.0 lazy-loading asOf() prevents corruption
|
||||
* **Consumer blob integrity**: Verified v5.4.0 lazy-loading asOf() prevents corruption
|
||||
- HistoricalStorageAdapter eliminates race conditions
|
||||
- Snapshots created on-demand (no commit-time snapshot)
|
||||
- Verified with 570-entity test matching Workshop production scale
|
||||
- Verified with 570-entity test matching consumer production scale
|
||||
|
||||
### ⚡ Performance Adjustments
|
||||
|
||||
|
|
@ -1912,7 +1912,7 @@ v5.1.0 delivers a significantly improved developer experience:
|
|||
- **Problem**: In v5.0.0, `saveNoun()` was called before `saveNounMetadata()`, causing TypeAwareStorage to default entity types to 'thing' and save to wrong storage paths
|
||||
- **Impact**: Broke VFS file operations, `brain.get()`, `brain.relate()`, and all features depending on entity metadata
|
||||
- **Solution**: Reversed save order - now saves metadata FIRST, then noun vector
|
||||
- **Fixes**: [Workshop Bug Report](https://github.com/soulcraftlabs/brain-cloud/issues/VFS-METADATA-MISSING)
|
||||
- **Fixes**: VFS metadata-missing regression (internal tracker)
|
||||
|
||||
**Fork API: Lazy COW Initialization**
|
||||
|
||||
|
|
@ -1939,7 +1939,7 @@ v5.1.0 delivers a significantly improved developer experience:
|
|||
|
||||
### 📊 Impact
|
||||
|
||||
* **Unblocks**: Workshop team and all VFS users
|
||||
* **Unblocks**: a consumer team and all VFS users
|
||||
* **Fixes**: All metadata-dependent features (get, relate, find, VFS)
|
||||
* **Maintains**: Full backward compatibility with v4.x data
|
||||
|
||||
|
|
@ -2287,7 +2287,7 @@ This release transforms Brainy imports from entity extractors into true knowledg
|
|||
- **Relationship Type Metadata**: All relationships tagged as `vfs`, `semantic`, or `provenance` for filtering
|
||||
- **Enhanced Column Detection**: 7 relationship types (vs 1 previously) - Location, Owner, Creator, Uses, Member, Friend, Related
|
||||
- **Type-Based Inference**: Smart relationship classification based on entity types and context analysis
|
||||
- **Impact**: Workshop import now creates ~3,900 relationships (vs 581), with 5-20+ connections per entity
|
||||
- **Impact**: A consumer import now creates ~3,900 relationships (vs 581), with 5-20+ connections per entity
|
||||
|
||||
* **import**: New configuration option `createProvenanceLinks` (defaults to `true`)
|
||||
- Enables/disables provenance relationship creation
|
||||
|
|
@ -2329,7 +2329,7 @@ Graph: Rich network, 5-20+ connections per entity
|
|||
|
||||
### [4.7.4](https://github.com/soulcraftlabs/brainy/compare/v4.7.3...v4.7.4) (2025-10-27)
|
||||
|
||||
**CRITICAL SYSTEMIC VFS BUG FIX - Workshop Team Unblocked!**
|
||||
**CRITICAL SYSTEMIC VFS BUG FIX - A consumer team Unblocked!**
|
||||
|
||||
This hotfix resolves a systemic bug affecting ALL storage adapters that caused VFS queries to return empty results even when data existed.
|
||||
|
||||
|
|
@ -2341,7 +2341,7 @@ This hotfix resolves a systemic bug affecting ALL storage adapters that caused V
|
|||
- **Bug Pattern**: `if (!metadata) continue` in getNouns()/getVerbs() methods
|
||||
- **Fixed Locations**: 12 bug sites across 7 adapters (TypeAware, Memory, FileSystem, GCS, S3, R2, OPFS, Azure)
|
||||
- **Solution**: Allow optional metadata with `metadata: (metadata || {}) as NounMetadata`
|
||||
- **Result**: Workshop team UNBLOCKED - VFS entities now queryable
|
||||
- **Result**: a consumer team UNBLOCKED - VFS entities now queryable
|
||||
|
||||
* **neural**: Fix SmartExtractor weighted score threshold bug (28 test failures → 4)
|
||||
- **Root Cause**: Single signal with 0.8 confidence × 0.2 weight = 0.16 < 0.60 threshold
|
||||
|
|
@ -2409,7 +2409,7 @@ Clean separation between VFS (Virtual File System) entities and knowledge graph
|
|||
### 🐛 Critical Bug Fixes
|
||||
|
||||
* **vfs.initializeRoot()**: add includeVFS to prevent duplicate root creation
|
||||
- **Critical Fix**: VFS init was creating ~10 duplicate root entities (Workshop team issue)
|
||||
- **Critical Fix**: VFS init was creating ~10 duplicate root entities (a consumer team issue)
|
||||
- **Root Cause**: `initializeRoot()` called `brain.find()` without `includeVFS: true`, never found existing VFS root
|
||||
- **Impact**: Every `vfs.init()` created a new root, causing empty `readdir('/')` results
|
||||
- **Solution**: Added `includeVFS: true` to root entity lookup (line 171)
|
||||
|
|
@ -2503,7 +2503,7 @@ const files = await vfs.search('documentation')
|
|||
- **Root Cause**: HNSW `rebuild()` and Graph `rebuild()` methods still called `getNounsWithPagination()`/`getVerbsWithPagination()` repeatedly
|
||||
- Each pagination call triggered `getAllShardedFiles()` reading all 256 shard directories
|
||||
- For 1,157 entities: MetadataIndex (2-3s) + HNSW (~20s) + Graph (~10s) = **30-35 seconds total**
|
||||
- Workshop team reported: "v4.2.3 is at batch 7 after ~60 seconds" - still far from claimed 100x improvement
|
||||
- a consumer team reported: "v4.2.3 is at batch 7 after ~60 seconds" - still far from claimed 100x improvement
|
||||
- **Solution**: Apply v4.2.3 adaptive loading pattern to ALL 3 indexes
|
||||
- **FileSystemStorage/MemoryStorage/OPFSStorage**: Load all entities at once (limit: 10000000)
|
||||
- **Cloud storage (GCS/S3/R2/Azure)**: Keep pagination (native APIs are efficient)
|
||||
|
|
@ -2524,7 +2524,7 @@ const files = await vfs.search('documentation')
|
|||
- Graph Index: Loads all verbs at once for local, paginated for cloud (lines 300-361)
|
||||
- Pattern matches v4.2.3 MetadataIndex implementation exactly
|
||||
- Zero config: Completely automatic based on storage adapter type
|
||||
- **Resolution**: Fully resolves Workshop team's v4.2.x performance regression
|
||||
- **Resolution**: Fully resolves a consumer team's v4.2.x performance regression
|
||||
- **Files Changed**:
|
||||
- `src/hnsw/hnswIndex.ts` (updated rebuild() with adaptive loading)
|
||||
- `src/graph/graphAdjacencyIndex.ts` (updated rebuild() with adaptive loading)
|
||||
|
|
@ -2549,7 +2549,7 @@ const files = await vfs.search('documentation')
|
|||
- Pagination was designed for cloud storage socket exhaustion
|
||||
- FileSystem doesn't need pagination - can handle loading thousands of entities at once
|
||||
- Eliminates repeated directory scans: 3 batches × 256 dirs → 1 batch × 256 dirs
|
||||
- **Workshop Team**: This resolves the v4.2.2 stalling issue - rebuild will now complete in seconds
|
||||
- **A consumer team**: This resolves the v4.2.2 stalling issue - rebuild will now complete in seconds
|
||||
- **Files Changed**: `src/utils/metadataIndex.ts` (rebuilt() method with adaptive loading strategy)
|
||||
|
||||
### [4.2.2](https://github.com/soulcraftlabs/brainy/compare/v4.2.1...v4.2.2) (2025-10-23)
|
||||
|
|
@ -2591,7 +2591,7 @@ const files = await vfs.search('documentation')
|
|||
- Universal: All storage adapters (FileSystem, GCS, S3, R2, Memory, OPFS)
|
||||
- **Zero Config**: Completely automatic, no configuration needed
|
||||
- **Self-Healing**: Gracefully handles missing/corrupt registry (rebuilds once)
|
||||
- **Impact**: Fixes Workshop team bug report - production-ready at billion scale
|
||||
- **Impact**: Fixes a consumer team bug report - production-ready at billion scale
|
||||
- **Files Changed**: `src/utils/metadataIndex.ts` (added saveFieldRegistry/loadFieldRegistry methods, updated init/flush)
|
||||
|
||||
### [4.2.0](https://github.com/soulcraftlabs/brainy/compare/v4.1.4...v4.2.0) (2025-10-23)
|
||||
|
|
@ -2635,7 +2635,7 @@ const files = await vfs.search('documentation')
|
|||
- Fixed property access bugs: `verb.target` → `verb.to`, `verb.verb` → `verb.type`
|
||||
- Added comprehensive integration tests (14 tests covering all query patterns)
|
||||
- Updated JSDoc documentation with usage examples
|
||||
- **Impact**: Resolves Workshop team bug where 524 imported relationships were inaccessible
|
||||
- **Impact**: Resolves a consumer team bug where 524 imported relationships were inaccessible
|
||||
- **Breaking**: None - fully backward compatible
|
||||
|
||||
### [4.1.2](https://github.com/soulcraftlabs/brainy/compare/v4.1.1...v4.1.2) (2025-10-21)
|
||||
|
|
|
|||
18
CLAUDE.md
18
CLAUDE.md
|
|
@ -165,6 +165,24 @@ After a successful release, remind the user:
|
|||
|
||||
Do NOT deploy portal from here. Portal is always deployed separately from within the portal project.
|
||||
|
||||
## Closed-Source Product Names — HARD RULE
|
||||
|
||||
Brainy is the only Soulcraft open-source project. Nothing in this repo — code, JSDoc, tests,
|
||||
docs, RELEASES.md, CHANGELOG.md, commit messages — may reference closed-source Soulcraft
|
||||
products by name (Workshop, Venue, Memory, Muse, Hall, Forge, Academy, Pulse, Heart,
|
||||
Collective, SDK) or by their specific class/method names (`BookingDraftService`,
|
||||
`getDemandHeatmap`, `systemKind`, etc.).
|
||||
|
||||
When recording a consumer-reported bug, regression scenario, or release note:
|
||||
- Refer to "a consumer", "a downstream application", "a production deployment", or "an
|
||||
internal report" — never name the product.
|
||||
- All doc examples must use generic domain values (`'employee'`, `'customer'`, `'invoice'`,
|
||||
`'milestone'`, `OrderService`, `/orders/...`), not product-specific schemas.
|
||||
- Internal session artifacts (`.strategy/`, `~/.claude/plans/`, handoff files outside the
|
||||
repo) MAY name products — those are not public.
|
||||
|
||||
If you catch yourself typing a product name into a tracked file, stop and rephrase.
|
||||
|
||||
## Performance Claims
|
||||
|
||||
When documenting performance characteristics:
|
||||
|
|
|
|||
155
RELEASES.md
155
RELEASES.md
|
|
@ -1,13 +1,144 @@
|
|||
# @soulcraft/brainy — Release Notes for Consumers
|
||||
|
||||
This file is the **quick reference for Soulcraft product sessions** tracking Brainy changes.
|
||||
This file is the **quick reference for downstream sessions** tracking Brainy changes.
|
||||
Full auto-generated changelog: `CHANGELOG.md` · Releases: https://github.com/soulcraftlabs/brainy/releases
|
||||
|
||||
**How to use:** Brainy is the underlying data engine for Workshop, Venue, Academy, and
|
||||
Collective. The SDK wraps it — most products never call Brainy directly. Read this when:
|
||||
- Upgrading `@soulcraft/brainy` in the SDK or a product
|
||||
**How to use:** Brainy is the underlying data engine for downstream applications. Read this when:
|
||||
- Upgrading `@soulcraft/brainy` in your application
|
||||
- Debugging data, query, or storage behaviour
|
||||
- A new Brainy feature is available that SDK should expose
|
||||
- A new Brainy feature is available that you want to adopt
|
||||
|
||||
---
|
||||
|
||||
## v7.30.2 — 2026-06-08
|
||||
|
||||
**Affected products:** any consumer with `find({ limit })` call sites that pass values
|
||||
≥ ~9000 — a common safety-cap pattern in production code (`limit: 10_000` against
|
||||
type-filtered queries that typically return 10–500 entities). Additive; drop-in
|
||||
from 7.30.1.
|
||||
|
||||
### Why
|
||||
|
||||
Brainy 7.30 introduced a memory-derived cap on `find({ limit })` to prevent OOM
|
||||
from runaway queries. The cap was sound in intent but **~4× too conservative in
|
||||
calibration**: the formula assumed 100 KB per result while typical entity footprint
|
||||
is 7-10 KB (384-dim float32 vector ≈ 1.5 KB + standard fields + metadata). On a 900 MB
|
||||
free-memory box this capped `limit` at 9000, breaking production dashboards where
|
||||
cascading 500s from queries with `limit: 10_000` silently degraded the `Promise.all`
|
||||
loading the canvas.
|
||||
|
||||
7.30.2 recalibrates the formula and changes the enforcement from synchronous-throw
|
||||
to two-tier (warn-then-throw) so existing pre-7.30 code doesn't break on upgrade.
|
||||
|
||||
### Recalibrated formula — `25 KB` per result (was `100 KB`)
|
||||
|
||||
The auto-configured cap now uses a realistic per-result size:
|
||||
|
||||
| Memory source | 7.30.0–7.30.1 cap | 7.30.2 cap |
|
||||
|---|---|---|
|
||||
| 4 GB Cloud Run container | 10 000 | 40 000 |
|
||||
| 2 GB container | 5 000 | 20 000 |
|
||||
| 900 MB free system memory | 9 000 | ~36 000 |
|
||||
| `reservedQueryMemory: 1 GB` | 10 000 | 40 000 |
|
||||
|
||||
Hard ceiling stays at 100 000. Existing `BrainyConfig.maxQueryLimit` /
|
||||
`reservedQueryMemory` overrides continue to work unchanged.
|
||||
|
||||
### Two-tier enforcement — warn, then throw
|
||||
|
||||
**Below cap (`limit ≤ maxLimit`):** silent pass. No signal, no friction. Unchanged.
|
||||
|
||||
**Soft tier (`maxLimit < limit ≤ 2 × maxLimit`)** *NEW*: one-time warning per call
|
||||
site (dedup keyed on stack location + limit value), query proceeds. Pre-7.30.2 code
|
||||
that relied on the cap silently allowing safety-cap limits keeps working — the
|
||||
warning teaches the recipe so consumers can fix it intentionally.
|
||||
|
||||
**Hard tier (`limit > 2 × maxLimit`)** *NEW*: throw with the same message format the
|
||||
warning uses. Real OOM territory; the cap stops being a recommendation and becomes
|
||||
a guardrail.
|
||||
|
||||
The 2× soft margin is chosen to absorb existing safety-cap patterns (`limit: 10_000`
|
||||
against a 9 K-cap box) without disabling OOM protection. Real OOM territory on a JS
|
||||
in-memory brain is hundreds of thousands of results, not 10× the safety cap.
|
||||
|
||||
### Improved error / warning message (mirrors the 7.30.1 enforcement-error format)
|
||||
|
||||
Before (7.30 → 7.30.1):
|
||||
```
|
||||
limit exceeds auto-configured maximum of 9000 (based on available memory)
|
||||
```
|
||||
|
||||
After (7.30.2):
|
||||
```
|
||||
find({ limit: 10000 }) exceeds the auto-configured query limit of 9000 (basis:
|
||||
available free memory). Choose one:
|
||||
• Increase the cap: new Brainy({ maxQueryLimit: 10000 })
|
||||
• Reserve more memory: new Brainy({ reservedQueryMemory: 256000000 })
|
||||
• Paginate: split the query with { limit, offset } pages
|
||||
at OrderService.loadDashboard (/app/src/orders/dashboard.ts:142:18)
|
||||
Docs: https://soulcraft.com/docs/guides/find-limits
|
||||
```
|
||||
|
||||
Caller location comes from the same `findCallerLocation()` helper introduced in
|
||||
7.30.1 (now extracted to `src/utils/callerLocation.ts` so both subtype enforcement
|
||||
and limit enforcement share it).
|
||||
|
||||
### New docs
|
||||
|
||||
New `docs/guides/subtypes-and-facets.md`-style guide at
|
||||
`docs/guides/find-limits.md` (public, indexed) — explains the cap, the four memory
|
||||
sources the auto-config considers, the three escape valves (`maxQueryLimit`,
|
||||
`reservedQueryMemory`, pagination), and when to use which. Explicit "pagination is
|
||||
the future-proof pattern" callout — the cap can get tighter in 8.0 (Datomic-style
|
||||
`Db.find()` may make per-call limits stricter to keep snapshot semantics cheap),
|
||||
but pagination keeps working unchanged.
|
||||
|
||||
`docs/api/README.md` `find()` entry gets a one-paragraph `limit` tip + pointer
|
||||
to the new guide.
|
||||
|
||||
### Tests
|
||||
|
||||
- **New `tests/integration/find-limits.test.ts`** (9 tests): below-cap silent
|
||||
pass; soft-tier warns once per call site (dedup verified by exercising same vs.
|
||||
different source lines); soft-tier message format (names all three escape
|
||||
valves + docs link); soft-tier message includes caller location; hard-tier
|
||||
throws; hard-tier message format same as soft-tier; consumer `maxQueryLimit`
|
||||
override raises the cap and shifts both tiers accordingly; **the pre-7.30.2
|
||||
regression scenario** explicitly covered — `limit: 10_000` against a
|
||||
`maxQueryLimit: 9000` cap now warns and passes instead of throwing.
|
||||
- Existing unit tests in `tests/unit/utils/memoryLimits.test.ts` updated to
|
||||
reflect the recalibrated formula (5 tests: hardcoded values for
|
||||
container / reserved / free-memory paths bumped 4×).
|
||||
- Existing `paramValidation.test.ts` "should auto-limit based on system memory"
|
||||
test extended to cover the new three-tier semantics (below-cap pass /
|
||||
soft-tier silent / hard-tier throw).
|
||||
- All prior suites still green: subtype-and-facets 26/26, verb-subtype-and-
|
||||
enforcement 30/30, strict-mode-self-test 13/13. Unit 1468/1468.
|
||||
|
||||
### Cortex compatibility
|
||||
|
||||
**No Cortex changes required for 7.30.2.** Every change is JS-side: formula
|
||||
recalibration runs in `ValidationConfig.constructor()`, two-tier enforcement runs
|
||||
in `validateFindParams()`, both fire before any storage/index/Cortex call. Cortex
|
||||
2.x and the pending Cortex 3.0 work both stay compatible without code changes.
|
||||
|
||||
The new `docs/guides/find-limits.md` calls out that 8.0's Datomic-style `Db.find()`
|
||||
may tighten per-call limits; that's a Brainy 8.0 / Cortex 3.0 coordination point
|
||||
documented in `.strategy/BRAINY-8.0-SUBTYPE-CONTRACT.md` (open question #4 covers
|
||||
the rollout staging, which now also applies to query limits).
|
||||
|
||||
### What consumers should do
|
||||
|
||||
- **If you've been carrying a `limit: 9000` workaround for the 7.30.0–7.30.1
|
||||
cap:** drop it on bump to 7.30.2 — existing `limit: 10_000` patterns now
|
||||
warn (teaching the recipe) but no longer throw. The warning is
|
||||
one-time-per-call-site, so production logs don't spam.
|
||||
- **All other consumers:** no action required if no enforcement-error was being
|
||||
hit. If you see new `[Brainy]` warnings in logs after upgrade, follow the
|
||||
recipe in the warning or read `docs/guides/find-limits.md`.
|
||||
- **SDK wrappers:** wrapper-level pools that auto-construct `Brainy` instances
|
||||
should consider surfacing `maxQueryLimit` / `reservedQueryMemory` as
|
||||
consumer-facing config; Brainy now documents the knob explicitly.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -72,8 +203,8 @@ add(): NounType.Event requires subtype but got undefined. Register vocabulary vi
|
|||
After:
|
||||
```
|
||||
add(): NounType.event requires subtype but got undefined.
|
||||
at BookingDraftService.getOrCreateByToken (/app/src/booking/draft.ts:42:23)
|
||||
Pass one of: booking, session, milestone.
|
||||
at OrderService.getOrCreateByToken (/app/src/orders/draft.ts:42:23)
|
||||
Pass one of: standard, recurring, milestone.
|
||||
Migration recipe: https://soulcraft.com/docs/guides/subtypes-and-facets#strict-mode
|
||||
```
|
||||
|
||||
|
|
@ -171,7 +302,7 @@ adds a strict-mode tip to the `add()` / `relate()` reference entries.
|
|||
### Tests
|
||||
|
||||
- **New `tests/integration/strict-mode-self-test.test.ts`** (13 tests): creates a brain under
|
||||
the exact SDK_CORE_VOCABULARY shape Venue hit + brain-wide strict mode, then exercises every
|
||||
a realistic consumer vocabulary shape + brain-wide strict mode, then exercises every
|
||||
internal Brainy path (VFS root/mkdir/writeFile/cp/mv/ln, aggregation engine, audit
|
||||
diagnostic, error-message UX). Zero rejections expected.
|
||||
- Existing 7.30.0 + 7.29.0 integration suites unchanged: 26/26 + 30/30.
|
||||
|
|
@ -658,7 +789,7 @@ open of a directory poisoned by 7.20.0/7.21.0 fixes it.
|
|||
7.21.0 added new storage-adapter methods (`supportsMultiProcessLocking`,
|
||||
`acquireWriterLock`, etc.) and called them unconditionally. Cortex 2.2.0's
|
||||
mmap storage adapter bundles a pre-7.21 `BaseStorage` and doesn't define
|
||||
them, so Venue's 7.21.0 upgrade attempt threw
|
||||
them, so a consumer's 7.21.0 upgrade attempt threw
|
||||
`TypeError: this.storage.supportsMultiProcessLocking is not a function`
|
||||
at boot.
|
||||
|
||||
|
|
@ -702,7 +833,7 @@ internal platform handoff.
|
|||
|
||||
## v7.21.0 — 2026-05-15
|
||||
|
||||
**Affected products:** All consumers using filesystem storage (Venue, Workshop dev,
|
||||
**Affected products:** All consumers using filesystem storage (production deploys,
|
||||
local development).
|
||||
|
||||
### Multi-process safety + first-class read-only inspector
|
||||
|
|
@ -807,7 +938,7 @@ and Cortex. Read-only inspectors mmap Cortex segments with zero coordination.
|
|||
|
||||
## v7.19.10 — 2026-02-24
|
||||
|
||||
**Affected products:** All Bun/ESM consumers (Workshop, Venue, Academy, SDK)
|
||||
**Affected products:** All Bun/ESM consumers
|
||||
|
||||
### ESM crypto fix in SSTable
|
||||
|
||||
|
|
@ -834,7 +965,7 @@ No API changes. If you were seeing ghost results in filtered queries, this fixes
|
|||
|
||||
## v7.18.0 — 2026-02-16
|
||||
|
||||
**Affected products:** Workshop, Venue, Academy (analytics, reporting, session summaries)
|
||||
**Affected products:** Analytics, reporting, and session-summary consumers
|
||||
|
||||
### Aggregation engine
|
||||
|
||||
|
|
|
|||
|
|
@ -251,6 +251,8 @@ const results = await brain.find({
|
|||
- `hybridAlpha?`: `number` - Balance between text (0.0) and semantic (1.0) search. Auto-detected by query length if not specified.
|
||||
- `excludeVFS?`: `boolean` - Exclude VFS entities from results (default: false)
|
||||
|
||||
> **`limit` tip:** Brainy caps `limit` against an auto-configured maximum (based on container/free memory, ~25 KB per result). Above the cap you get a one-time warning per call site; above 2× the cap it throws. To raise the cap, pass `new Brainy({ maxQueryLimit: N })` or `{ reservedQueryMemory: bytes }`. For queries that need ALL matches, paginate with `{ limit, offset }` — that's the only pattern guaranteed to keep working across Brainy versions. See [Query Limits & Pagination](../guides/find-limits.md).
|
||||
|
||||
**Returns:** `Promise<Result[]>` - Matching entities with scores
|
||||
|
||||
---
|
||||
|
|
|
|||
10
docs/eli5.md
10
docs/eli5.md
|
|
@ -150,12 +150,6 @@ Add Cortex and you also unlock memory-mapped storage — aggregate state lives d
|
|||
- **Safe experiments** — Let teams branch the knowledge base, experiment independently, and merge when ready — just like branching code.
|
||||
- **Unified business platforms** — Combine booking, CRM, inventory, and analytics in one queryable knowledge graph with no sync pipeline.
|
||||
|
||||
### Built with Brainy
|
||||
### What Brainy is good at
|
||||
|
||||
Real products built on Brainy — live in production at Soulcraft:
|
||||
|
||||
- **Workshop** — AI-powered IDE and creation studio where imagination meets creation.
|
||||
- **Venue** — Unified physical + digital business operations, from food truck to franchise.
|
||||
- **Memory** — Infinite context for AI agents that never forgets.
|
||||
- **Collective** — Multi-agent coordination with shared knowledge and automatic task routing.
|
||||
- **Heart** — Emotional intelligence and empathy layer for AI communication.
|
||||
Brainy is the engine underneath production systems that need to combine semantic search, structured filtering, and graph traversal in a single query — agent memory, knowledge-base platforms, business operations consoles, multi-agent coordination, and more. The combination of vector + graph + metadata search in one indexed call is what differentiates it from running three engines side by side.
|
||||
|
|
|
|||
149
docs/guides/find-limits.md
Normal file
149
docs/guides/find-limits.md
Normal file
|
|
@ -0,0 +1,149 @@
|
|||
---
|
||||
title: Query Limits & Pagination
|
||||
slug: guides/find-limits
|
||||
public: true
|
||||
category: guides
|
||||
template: guide
|
||||
order: 8
|
||||
description: How Brainy caps `find({ limit })` to prevent OOM, the three escape valves when the cap is too tight, and why pagination is the future-proof pattern.
|
||||
next:
|
||||
- guides/aggregation
|
||||
- api/reference
|
||||
---
|
||||
|
||||
# Query Limits & Pagination
|
||||
|
||||
Brainy's `find()` returns entities into a JavaScript array. The size of that array is bounded by an auto-configured cap so a single query can never run the host out of memory. This guide explains the cap, the three ways to raise it when your use case justifies it, and the one pattern that scales no matter what cap is in effect: pagination.
|
||||
|
||||
## Why the cap exists
|
||||
|
||||
Every entity Brainy returns carries:
|
||||
|
||||
- A 384-dim float32 embedding vector (1.5 KB)
|
||||
- Standard fields: `id`, `type`, `subtype`, timestamps, confidence, weight (~200 bytes)
|
||||
- User metadata (variable — typical 5-10 KB, can spike to 20+ KB)
|
||||
|
||||
Conservative budget: **25 KB per result**. A `find({ limit: 100_000 })` against a brain with rich metadata can claim ~2.5 GB before Brainy's iteration starts. JavaScript's GC + V8's heap targets can't absorb that swing without paging or OOM in production.
|
||||
|
||||
The cap is a safety net. It's not the only reason your query might be slow — graph traversal and HNSW search have their own perf characteristics — but it's the one that turns a slow query into a sudden runtime error.
|
||||
|
||||
## The auto-configured cap (7.30.2+)
|
||||
|
||||
Brainy picks `maxLimit` from the first of these that's available:
|
||||
|
||||
| Priority | Source | Formula |
|
||||
|---|---|---|
|
||||
| 1 | Constructor option `maxQueryLimit` | Hard cap at supplied value, max 100 000 |
|
||||
| 2 | Constructor option `reservedQueryMemory` | `floor(reservedQueryMemory / 25 KB)` capped at 100 000 |
|
||||
| 3 | Detected container memory limit (Cloud Run, Kubernetes, cgroups v1/v2) | `floor(containerLimit × 0.25 / 25 KB)` capped at 100 000 |
|
||||
| 4 | Free system memory | `floor(availableMemory / 25 KB)` capped at 100 000 |
|
||||
|
||||
Worked example: a 4 GB Cloud Run container picks priority 3 → `floor(4 GB × 0.25 / 25 KB) = floor(40 960) = 40 000` results. A 900 MB free-memory box on priority 4 gets `floor(900 MB / 25 KB) = ~36 000`.
|
||||
|
||||
> **Calibration note.** Pre-7.30.2 used 100 KB per result instead of 25 KB, which produced caps that were 4× too tight for typical workloads (an 8 KB / result reality). 7.30.2 recalibrated to match observed entity sizes; existing `limit: 10_000` safety patterns now pass silently on any reasonably-sized box.
|
||||
|
||||
## What happens when you exceed the cap
|
||||
|
||||
`find({ limit })` enforces in **two tiers**:
|
||||
|
||||
### Soft tier: `maxLimit < limit ≤ 2 × maxLimit`
|
||||
|
||||
You get a one-time warning per call site:
|
||||
|
||||
```
|
||||
[Brainy] find({ limit: 50000 }) exceeds the auto-configured query limit of
|
||||
40000 (basis: detected container memory limit). Choose one:
|
||||
• Increase the cap: new Brainy({ maxQueryLimit: 50000 })
|
||||
• Reserve more memory: new Brainy({ reservedQueryMemory: 1310720000 })
|
||||
• Paginate: split the query with { limit, offset } pages
|
||||
at YourService.loadDashboard (/app/src/dashboard.ts:142:18)
|
||||
Docs: https://soulcraft.com/docs/guides/find-limits
|
||||
```
|
||||
|
||||
**The query proceeds.** Brainy returns the result set you asked for; the warning is a teaching signal, not a block. Existing code that relied on the cap silently allowing safety-cap limits (`limit: 10_000` against a 9 K-cap box) keeps working — the warning shows you the recipe so you can fix it intentionally.
|
||||
|
||||
### Hard tier: `limit > 2 × maxLimit`
|
||||
|
||||
Same message, but thrown as an error. This is real OOM territory; the cap stops being a recommendation and becomes a guardrail.
|
||||
|
||||
## The three escape valves
|
||||
|
||||
### 1. Raise the cap at construction — `maxQueryLimit`
|
||||
|
||||
When the auto-config is wrong for your workload (e.g. you know your entities are smaller than 25 KB average and you need bigger result sets), set an explicit cap:
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'filesystem', options: { path: './data' } },
|
||||
maxQueryLimit: 50_000 // raises the cap; still hard-clamped at 100 000
|
||||
})
|
||||
```
|
||||
|
||||
This is the right answer when:
|
||||
- Your entity metadata is genuinely small (e.g. 1-2 KB) and 25 KB per result is over-conservative
|
||||
- You're running on a box with lots of headroom and 25% of memory underestimates what you can spare for queries
|
||||
- You need a known-good limit that doesn't change when the box's free-memory wiggles at startup
|
||||
|
||||
### 2. Reserve more memory for queries — `reservedQueryMemory`
|
||||
|
||||
When you want the cap to be memory-derived but more generous than the default 25% slice:
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
reservedQueryMemory: 1024 * 1024 * 1024 // 1 GB → ~40 000 result cap
|
||||
})
|
||||
```
|
||||
|
||||
This is the right answer when:
|
||||
- Your host's memory budget for queries is known and stable, regardless of free-memory at startup
|
||||
- You want the formula to scale with the documented per-result size (25 KB) instead of a hard number
|
||||
|
||||
### 3. Paginate — the future-proof pattern
|
||||
|
||||
If your query genuinely needs to walk all matches in a category, don't fight the cap — walk in pages:
|
||||
|
||||
```typescript
|
||||
async function findAll<T>(params: FindParams<T>, pageSize = 1000): Promise<Result<T>[]> {
|
||||
const all: Result<T>[] = []
|
||||
let offset = 0
|
||||
while (true) {
|
||||
const page = await brain.find({ ...params, limit: pageSize, offset })
|
||||
all.push(...page)
|
||||
if (page.length < pageSize) break
|
||||
offset += page.length
|
||||
}
|
||||
return all
|
||||
}
|
||||
|
||||
// Use it just like find():
|
||||
const allEvents = await findAll({ type: NounType.Event, where: { status: 'open' } })
|
||||
```
|
||||
|
||||
For very large brains, prefer the streaming API which avoids holding the full result set in memory at all:
|
||||
|
||||
```typescript
|
||||
for await (const entity of brain.streaming.entities({ type: NounType.Event })) {
|
||||
// process one entity at a time
|
||||
}
|
||||
```
|
||||
|
||||
## When to use which
|
||||
|
||||
| Situation | Recommended valve |
|
||||
|---|---|
|
||||
| The cap is unreasonably low for your known entity size | `maxQueryLimit` |
|
||||
| You want a memory-derived cap but more generous than 25% | `reservedQueryMemory` |
|
||||
| Your query needs ALL matches in a category | Pagination or `brain.streaming.entities()` |
|
||||
| You hit the cap once during a one-off migration | `maxQueryLimit` or `migrateField` (which already paginates internally) |
|
||||
| You're hitting the cap on a recurring user-facing query | Pagination — the cap will get tighter in 8.0, not looser |
|
||||
|
||||
## A note on Brainy 8.0
|
||||
|
||||
8.0's Datomic-style `Db` API may make per-call limits stricter to keep snapshot semantics cheap. **Pagination is the only pattern that's guaranteed to keep working unchanged.** Code that paginates today doesn't need to revisit when 8.0 ships.
|
||||
|
||||
## Reference
|
||||
|
||||
- `BrainyConfig.maxQueryLimit?: number` — explicit cap override (max 100 000)
|
||||
- `BrainyConfig.reservedQueryMemory?: number` — memory budget for queries (bytes)
|
||||
- `find({ limit, offset })` — paginated find
|
||||
- `brain.streaming.entities(filter)` — streaming alternative for very large traversals
|
||||
|
|
@ -464,7 +464,7 @@ await brain.add({ type: NounType.Person, subtype: 'employee', data: '...' })
|
|||
|
||||
When a platform layer like the Soulcraft SDK registers `requireSubtype()` rules on behalf of every consumer's brain, every downstream product that calls `brain.add()` / `brain.relate()` against those types must pass a matching `subtype`. Skipping the field — or passing one outside the registered vocabulary — throws at the boundary.
|
||||
|
||||
This pattern is powerful but surfaces a class of latent bug: any `brain.add()` call site that was written before strict-mode adoption starts rejecting writes. The Venue team hit this in production on 2026-06-08 when their `/book` flow 500'd on every request because `BookingDraftService.getOrCreateByToken` called `brain.add({ type: NounType.Event, ... })` without subtype.
|
||||
This pattern is powerful but surfaces a class of latent bug: any `brain.add()` call site that was written before strict-mode adoption starts rejecting writes. A representative production incident: a booking flow started returning 500 on every request because a service method called `brain.add({ type: NounType.Event, ... })` without subtype, and an SDK layer had just registered `requireSubtype()` for `NounType.Event` on every brain instance.
|
||||
|
||||
The fix is a four-step migration recipe — and Brainy 7.30.1+ ships diagnostic tools to make it deterministic.
|
||||
|
||||
|
|
@ -488,7 +488,7 @@ The fix is a four-step migration recipe — and Brainy 7.30.1+ ships diagnostic
|
|||
2. **Bulk-migrate any existing convention** with `brain.migrateField()` if a legacy field can be lifted:
|
||||
|
||||
```typescript
|
||||
// Venue chose subtype = same string as metadata.entityType:
|
||||
// Common pattern: subtype mirrors a discriminator field already in metadata
|
||||
await brain.migrateField({
|
||||
from: 'metadata.entityType',
|
||||
to: 'subtype',
|
||||
|
|
@ -496,7 +496,7 @@ The fix is a four-step migration recipe — and Brainy 7.30.1+ ships diagnostic
|
|||
})
|
||||
```
|
||||
|
||||
3. **Hand-fix the remaining call sites.** The exact list is in `report.entitiesWithoutSubtype`. For each call site, add `subtype: '<value>'` to the `brain.add()` / `brain.relate()` params. Choose a stable convention (Venue chose `subtype = metadata.entityType`; any rule that's deterministic from the data works).
|
||||
3. **Hand-fix the remaining call sites.** The exact list is in `report.entitiesWithoutSubtype`. For each call site, add `subtype: '<value>'` to the `brain.add()` / `brain.relate()` params. Choose a stable convention (e.g. mirror `metadata.entityType` if you have one; any rule that's deterministic from the data works).
|
||||
|
||||
4. **Verify with `brain.audit()` again.** Re-run; total should be `0`. If you turn on brain-wide strict mode at this point, all future writes are protected.
|
||||
|
||||
|
|
|
|||
|
|
@ -55,6 +55,7 @@ import {
|
|||
validateFindParams,
|
||||
recordQueryPerformance
|
||||
} from './utils/paramValidation.js'
|
||||
import { findCallerLocation } from './utils/callerLocation.js'
|
||||
import {
|
||||
SaveNounMetadataOperation,
|
||||
SaveNounOperation,
|
||||
|
|
@ -2773,7 +2774,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
strict: boolean
|
||||
}): string {
|
||||
const typeLabel = opts.kind === 'noun' ? 'NounType' : 'VerbType'
|
||||
const callSite = this.findCallerLocation()
|
||||
const callSite = findCallerLocation()
|
||||
const vocab = opts.rule?.values ? Array.from(opts.rule.values).join(', ') : null
|
||||
|
||||
let head: string
|
||||
|
|
@ -2803,28 +2804,9 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
return [head, callerLine, guidance, docLink].filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the first non-Brainy frame from the current stack so error messages
|
||||
* can point at the consumer's call site instead of Brainy internals. Returns
|
||||
* `null` if the stack isn't available (some runtimes) or only contains Brainy
|
||||
* frames.
|
||||
*/
|
||||
private findCallerLocation(): string | null {
|
||||
const stack = new Error().stack
|
||||
if (!stack) return null
|
||||
const lines = stack.split('\n').slice(1) // drop the `Error` line
|
||||
for (const raw of lines) {
|
||||
const line = raw.trim()
|
||||
// Skip frames inside Brainy's own files. We don't want to point the user
|
||||
// at `brainy.ts:XXXX` — they need their own call site.
|
||||
if (line.includes('/src/brainy.ts') || line.includes('/dist/brainy.js')) continue
|
||||
if (line.includes('enforceSubtypeOn') || line.includes('formatSubtypeError')) continue
|
||||
if (line.includes('findCallerLocation')) continue
|
||||
// Strip leading `at ` if present so the caller can format consistently.
|
||||
return line.replace(/^at /, '')
|
||||
}
|
||||
return null
|
||||
}
|
||||
// findCallerLocation is now exported from src/utils/callerLocation.ts so
|
||||
// both the subtype enforcement errors here and the query-limit warnings in
|
||||
// paramValidation.ts can share it without re-importing brainy.ts.
|
||||
|
||||
/**
|
||||
* Validate a metadata bag (or top-level field assignment) against any registered
|
||||
|
|
@ -4163,9 +4145,10 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
// Step 2: Copy storage ref (COW layer - instant!)
|
||||
await refManager.copyRef(currentBranch, branchName)
|
||||
|
||||
// CRITICAL FIX: Verify branch was actually created to prevent silent failures
|
||||
// Without this check, fork() could complete successfully but branch wouldn't exist,
|
||||
// causing subsequent checkout() calls to fail (see Workshop bug report).
|
||||
// CRITICAL FIX: Verify branch was actually created to prevent silent failures.
|
||||
// Without this check, fork() could complete successfully but the branch wouldn't
|
||||
// exist on disk, causing subsequent checkout() calls to fail with a
|
||||
// "Branch does not exist" error.
|
||||
const verifyBranch = await refManager.getRef(branchName)
|
||||
if (!verifyBranch) {
|
||||
throw new Error(
|
||||
|
|
@ -4553,7 +4536,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
* - Subsequent queries: Same as normal Brainy (uses rebuilt indexes)
|
||||
* - Memory overhead: Snapshot has separate in-memory indexes
|
||||
*
|
||||
* Use case: Workshop app - render file tree at historical commit
|
||||
* Use case: rendering a file tree (or any indexed view) at a historical commit
|
||||
*
|
||||
* @param commitId - Commit hash to snapshot from
|
||||
* @returns Read-only Brainy instance with historical state
|
||||
|
|
@ -5379,7 +5362,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
|
||||
/**
|
||||
* Extract entities from text (alias for extract())
|
||||
* Added for API clarity and Workshop team request
|
||||
* Added for API clarity — `extractEntities()` reads more naturally at call sites
|
||||
*
|
||||
* Uses NeuralEntityExtractor with SmartExtractor ensemble (4-signal architecture):
|
||||
* - ExactMatch (40%) - Dictionary lookups
|
||||
|
|
|
|||
|
|
@ -183,7 +183,7 @@ export const ACCURATE_PRESET: PresetConfig = {
|
|||
* - Fast, deterministic results
|
||||
* - Perfect for Excel/CSV with "Related Terms" columns
|
||||
*
|
||||
* Use case: Workshop glossary, structured taxonomies
|
||||
* Use case: structured taxonomies and glossaries from spreadsheet sources
|
||||
* Performance: ~5ms per row
|
||||
* Accuracy: ~99% (high confidence)
|
||||
*/
|
||||
|
|
@ -290,7 +290,7 @@ export function autoDetectPreset(context: ImportContext = {}): PresetConfig {
|
|||
}
|
||||
|
||||
// Rule 3: Structured data with explicit relationships → explicit preset
|
||||
// Perfect for Workshop bug fix!
|
||||
// (Handles spreadsheet imports where relationships are encoded in columns.)
|
||||
if (hasExplicitColumns && (fileType === 'excel' || fileType === 'csv')) {
|
||||
return EXPLICIT_PRESET
|
||||
}
|
||||
|
|
|
|||
|
|
@ -719,7 +719,7 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
|||
// CRITICAL FIX: COW metadata (_cow/*) must NEVER be branch-scoped
|
||||
// Refs, commits, and blobs are global metadata with their own internal branching.
|
||||
// Branch-scoping COW paths causes fork() to write refs to wrong locations,
|
||||
// leading to "Branch does not exist" errors on checkout (see Workshop bug report).
|
||||
// leading to "Branch does not exist" errors on checkout.
|
||||
if (basePath.startsWith('_cow/')) {
|
||||
return basePath // COW metadata is global across all branches
|
||||
}
|
||||
|
|
|
|||
54
src/utils/callerLocation.ts
Normal file
54
src/utils/callerLocation.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
/**
|
||||
* @module utils/callerLocation
|
||||
* @description Stack-frame extraction utility used by error/warning messages
|
||||
* across Brainy. The first non-Brainy frame on the stack is the consumer's
|
||||
* actual call site — the one that's useful to point at in diagnostics.
|
||||
*
|
||||
* Used by:
|
||||
* - Subtype enforcement errors in `brainy.ts` (7.30.1+) — guides consumers
|
||||
* from `requireSubtype()` rejections back to the offending call site.
|
||||
* - Query-limit enforcement in `paramValidation.ts` (7.30.2+) — pairs with
|
||||
* the recalibrated limit cap so consumers see which `find({ limit })`
|
||||
* call triggered the warn-or-throw.
|
||||
*
|
||||
* No runtime dependencies. Pure stack walking + string matching.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Extract the first non-Brainy frame from the current stack so error and
|
||||
* warning messages can point at the consumer's call site instead of Brainy
|
||||
* internals.
|
||||
*
|
||||
* The function walks `new Error().stack`, skips any frame inside Brainy's own
|
||||
* source or compiled output, and returns the first remaining frame stripped of
|
||||
* the leading `at ` token so the caller can compose the rendered line however
|
||||
* it likes.
|
||||
*
|
||||
* @param extraSkipPatterns Optional list of substrings — frames matching ANY
|
||||
* of these are also skipped. Use for in-module helpers that show up between
|
||||
* the public API and the consumer (e.g. the formatter that builds the
|
||||
* error message itself).
|
||||
* @returns The caller's location string (e.g. `"OrderService.create (/app/src/orders/service.ts:42:23)"`)
|
||||
* or `null` when the stack isn't available or only contains Brainy frames.
|
||||
*/
|
||||
export function findCallerLocation(extraSkipPatterns: string[] = []): string | null {
|
||||
const stack = new Error().stack
|
||||
if (!stack) return null
|
||||
const lines = stack.split('\n').slice(1) // drop the `Error` line
|
||||
for (const raw of lines) {
|
||||
const line = raw.trim()
|
||||
// Always skip Brainy's own source + compiled output. Consumers need their
|
||||
// own call site, not `brainy.ts:XXXX` or `dist/brainy.js:XXXX`.
|
||||
if (line.includes('/src/brainy.ts') || line.includes('/dist/brainy.js')) continue
|
||||
// Skip the validation + diagnostic helpers regardless of which file they
|
||||
// live in — they're plumbing between the public API and the consumer.
|
||||
if (line.includes('findCallerLocation')) continue
|
||||
if (line.includes('paramValidation') && line.includes('validate')) continue
|
||||
if (line.includes('enforceSubtypeOn') || line.includes('formatSubtypeError')) continue
|
||||
// Caller-supplied patterns (e.g. specific formatter names)
|
||||
if (extraSkipPatterns.some(p => line.includes(p))) continue
|
||||
// Strip leading `at ` if present so the caller can format consistently.
|
||||
return line.replace(/^at /, '')
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
|
@ -7,6 +7,8 @@
|
|||
|
||||
import { FindParams, AddParams, UpdateParams, RelateParams, UpdateRelationParams } from '../types/brainy.types.js'
|
||||
import { NounType, VerbType } from '../types/graphTypes.js'
|
||||
import { prodLog } from './logger.js'
|
||||
import { findCallerLocation } from './callerLocation.js'
|
||||
|
||||
// Dynamic import for Node.js os and fs modules
|
||||
let os: any = null
|
||||
|
|
@ -116,6 +118,43 @@ const getContainerMemoryLimit = (): number | null => {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Memory budget per query result, in KB. Used by the auto-configured
|
||||
* `maxLimit` formula across all three memory-derived priorities (reserved /
|
||||
* container / free).
|
||||
*
|
||||
* Calibration history:
|
||||
* - Pre-7.30.2: `100` (assumed 100 KB per result). Way too conservative —
|
||||
* actual entity footprint in Brainy is 7-10 KB (384-dim float32 vector ≈
|
||||
* 1.5 KB + standard fields + metadata). On a 900 MB free-memory box this
|
||||
* capped `limit` at 9_000, breaking common safety-cap call patterns like
|
||||
* `find({ type, where, limit: 10_000 })` that typically return 10-500
|
||||
* entities. Surfaced as `BR-MAXLIMIT-9000` in PLATFORM-HANDOFF.md.
|
||||
* - 7.30.2+: `25` (assumes 25 KB per result). Generous over typical (7-10 KB),
|
||||
* comfortably under the worst case (~20 KB with large metadata blobs).
|
||||
* Same 900 MB box now gives ~36_000 — typical 10_000 limits pass silently,
|
||||
* the warning tier surfaces around 36-72 K (encouraging pagination), and
|
||||
* the throw tier still fires before OOM territory (72 K+).
|
||||
*/
|
||||
const MAX_LIMIT_KB_PER_RESULT = 25
|
||||
|
||||
/**
|
||||
* One-time-per-call-site warning dedup. Keyed on the caller location returned
|
||||
* by `findCallerLocation()` plus the exceeding limit value so the warning fires
|
||||
* once per offending source line (not once per query). Survives the lifetime of
|
||||
* the process — that's intentional: the warning is a teaching signal, not a
|
||||
* recurring nag. Used by the two-tier limit enforcement in `validateFindParams`.
|
||||
*/
|
||||
const seenLimitWarnings = new Set<string>()
|
||||
|
||||
/**
|
||||
* Reset the limit-warning dedup. For tests that need a clean slate; production
|
||||
* code should never call this.
|
||||
*/
|
||||
export function resetLimitWarningCache(): void {
|
||||
seenLimitWarnings.clear()
|
||||
}
|
||||
|
||||
/**
|
||||
* Configuration options for ValidationConfig
|
||||
*/
|
||||
|
|
@ -174,7 +213,7 @@ export class ValidationConfig {
|
|||
if (options?.reservedQueryMemory !== undefined) {
|
||||
this.maxLimit = Math.min(
|
||||
100000,
|
||||
Math.floor(options.reservedQueryMemory / (1024 * 1024 * 100)) * 1000
|
||||
Math.floor(options.reservedQueryMemory / (1024 * 1024 * MAX_LIMIT_KB_PER_RESULT)) * 1000
|
||||
)
|
||||
this.limitBasis = 'reservedMemory'
|
||||
|
||||
|
|
@ -193,7 +232,7 @@ export class ValidationConfig {
|
|||
|
||||
this.maxLimit = Math.min(
|
||||
100000,
|
||||
Math.floor(queryMemory / (1024 * 1024 * 100)) * 1000
|
||||
Math.floor(queryMemory / (1024 * 1024 * MAX_LIMIT_KB_PER_RESULT)) * 1000
|
||||
)
|
||||
this.limitBasis = 'containerMemory'
|
||||
|
||||
|
|
@ -209,7 +248,7 @@ export class ValidationConfig {
|
|||
|
||||
this.maxLimit = Math.min(
|
||||
100000,
|
||||
Math.floor(availableMemory / (1024 * 1024 * 100)) * 1000
|
||||
Math.floor(availableMemory / (1024 * 1024 * MAX_LIMIT_KB_PER_RESULT)) * 1000
|
||||
)
|
||||
this.limitBasis = 'freeMemory'
|
||||
|
||||
|
|
@ -263,6 +302,95 @@ export class ValidationConfig {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Two-tier limit enforcement for `find({ limit })`. Compares the requested
|
||||
* limit against the auto-configured (or consumer-overridden) `maxLimit` and
|
||||
* picks one of three outcomes:
|
||||
*
|
||||
* 1. **Pass** — `limit <= maxLimit`. The configured cap was set with this
|
||||
* consumer's use case in mind; no signal, no friction.
|
||||
*
|
||||
* 2. **Warn** — `maxLimit < limit <= 2 * maxLimit`. The consumer is asking
|
||||
* for more than the cap, but not enough to be in OOM territory. Log a
|
||||
* one-time warning per call site (dedup keyed on stack location + limit
|
||||
* value) explaining the cap, the three escape valves, and the docs link.
|
||||
* The query proceeds. Pre-7.30.2 code that relied on the cap silently
|
||||
* allowing typical safety-cap limits (`10_000`) keeps working; the
|
||||
* warning teaches the migration.
|
||||
*
|
||||
* 3. **Throw** — `limit > 2 * maxLimit`. Real OOM danger territory. Throw
|
||||
* with the same message format the warning uses so the consumer gets
|
||||
* consistent guidance whether they hit the soft or hard threshold.
|
||||
*
|
||||
* The 2× soft margin is chosen to absorb existing safety-cap patterns
|
||||
* (`limit: 10_000` against a `maxLimit: 9_000` box, common case at 7.30 GA)
|
||||
* without disabling OOM protection. Real OOM territory on a JS in-memory
|
||||
* brain is hundreds of thousands of results, not 10× the safety cap.
|
||||
*
|
||||
* Both warning and throw paths use the same formatter so the rendered message
|
||||
* is identical — consumers see the same recipe regardless of which threshold
|
||||
* they crossed.
|
||||
*
|
||||
* @param limit - The requested `limit` value (already validated non-negative)
|
||||
* @param config - The active ValidationConfig (carries `maxLimit` + basis)
|
||||
* @throws When `limit > 2 * config.maxLimit`
|
||||
*
|
||||
* @since 7.30.2 (replaced the unconditional throw added in 7.30.0)
|
||||
*/
|
||||
function enforceLimitCap(limit: number, config: ValidationConfig): void {
|
||||
if (limit <= config.maxLimit) return
|
||||
|
||||
const message = formatLimitMessage(limit, config)
|
||||
const hardCeiling = config.maxLimit * 2
|
||||
|
||||
if (limit > hardCeiling) {
|
||||
// OOM danger zone — throw to prevent runaway memory allocation
|
||||
throw new Error(message)
|
||||
}
|
||||
|
||||
// Soft margin: warn once per call site + limit value, then let the query proceed.
|
||||
// Survives the lifetime of the process; the goal is to teach the migration recipe,
|
||||
// not to spam the log every query.
|
||||
const caller = findCallerLocation(['validateFindParams', 'enforceLimitCap', 'formatLimitMessage']) ?? '<unknown caller>'
|
||||
const dedupKey = `${caller}|${limit}`
|
||||
if (!seenLimitWarnings.has(dedupKey)) {
|
||||
seenLimitWarnings.add(dedupKey)
|
||||
prodLog.warn('[Brainy] ' + message)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the user-facing message used by both the warning tier and the throw
|
||||
* tier of `enforceLimitCap`. Same body shape as the 7.30.1 enforcement-error
|
||||
* messages: state the problem, name the recipe, point at the call site, link
|
||||
* the docs.
|
||||
*/
|
||||
function formatLimitMessage(limit: number, config: ValidationConfig): string {
|
||||
const cap = config.maxLimit
|
||||
const basis = config.limitBasis
|
||||
const basisLabel: Record<string, string> = {
|
||||
override: 'consumer-supplied `maxQueryLimit`',
|
||||
reservedMemory: 'consumer-supplied `reservedQueryMemory`',
|
||||
containerMemory: 'detected container memory limit',
|
||||
freeMemory: 'available free memory'
|
||||
}
|
||||
const basisText = basisLabel[basis] ?? 'available memory'
|
||||
const caller = findCallerLocation(['enforceLimitCap', 'formatLimitMessage'])
|
||||
|
||||
const lines = [
|
||||
`find({ limit: ${limit} }) exceeds the auto-configured query limit of ${cap} (basis: ${basisText}). Choose one:`,
|
||||
` • Increase the cap: new Brainy({ maxQueryLimit: ${Math.min(limit, 100000)} })`,
|
||||
` • Reserve more memory: new Brainy({ reservedQueryMemory: ${limit * MAX_LIMIT_KB_PER_RESULT * 1024} })`,
|
||||
' • Paginate: split the query with { limit, offset } pages'
|
||||
]
|
||||
if (caller) {
|
||||
lines.push(` at ${caller}`)
|
||||
}
|
||||
lines.push('Docs: https://soulcraft.com/docs/guides/find-limits')
|
||||
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Universal validations - things that are always invalid
|
||||
* These are mathematical/logical truths, not configuration
|
||||
|
|
@ -275,9 +403,7 @@ export function validateFindParams(params: FindParams): void {
|
|||
if (params.limit < 0) {
|
||||
throw new Error('limit must be non-negative')
|
||||
}
|
||||
if (params.limit > config.maxLimit) {
|
||||
throw new Error(`limit exceeds auto-configured maximum of ${config.maxLimit} (based on available memory)`)
|
||||
}
|
||||
enforceLimitCap(params.limit, config)
|
||||
}
|
||||
|
||||
if (params.offset !== undefined && params.offset < 0) {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/**
|
||||
* Integration tests for clear() bug fix (v5.10.4)
|
||||
*
|
||||
* Bug report: Workshop team reported that brain.clear() doesn't fully delete persistent storage.
|
||||
* Bug report: a consumer team reported that brain.clear() doesn't fully delete persistent storage.
|
||||
* After calling clear() and creating a new Brainy instance, all data was restored from _cow/ directory.
|
||||
*
|
||||
* Root cause: Setting cowEnabled = false on old instance doesn't affect new instances.
|
||||
|
|
@ -33,7 +33,7 @@ describe('Clear Persistence Bug Fix (v5.10.4)', () => {
|
|||
}
|
||||
})
|
||||
|
||||
it('should fully clear persistent storage (Workshop scenario)', async () => {
|
||||
it('should fully clear persistent storage (the reported scenario)', async () => {
|
||||
// Step 1: Create and populate instance
|
||||
const brain1 = new Brainy({
|
||||
storage: {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
/**
|
||||
* Integration tests for clear() VFS reinitialization fix (v7.3.1)
|
||||
*
|
||||
* Bug report: Workshop team reported that brain.clear() breaks VFS operations.
|
||||
* Bug report: a consumer team reported that brain.clear() breaks VFS operations.
|
||||
* After calling clear(), VFS operations fail with:
|
||||
* "Error: Source entity 00000000-0000-0000-0000-000000000000 not found"
|
||||
*
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
* commit objects on disk.
|
||||
*
|
||||
* Related bugs:
|
||||
* - Workshop team report: BRAINY_V5.3.0_SNAPSHOT_BUG_REPORT.md
|
||||
* - v5.3.0 snapshot regression filed via consumer bug report (BRAINY_V5.3.0_SNAPSHOT_BUG_REPORT.md, internal)
|
||||
* - Root cause: BlobStorage.ts hardcoded 'blob:' prefix in 9 locations
|
||||
*/
|
||||
|
||||
|
|
|
|||
191
tests/integration/find-limits.test.ts
Normal file
191
tests/integration/find-limits.test.ts
Normal file
|
|
@ -0,0 +1,191 @@
|
|||
/**
|
||||
* @module tests/integration/find-limits
|
||||
* @description Integration coverage for the 7.30.2 `find({ limit })` cap
|
||||
* recalibration + two-tier enforcement (warn-then-throw). See
|
||||
* `BR-MAXLIMIT-9000` in PLATFORM-HANDOFF.md for the original incident report
|
||||
* and `docs/guides/find-limits.md` for the consumer-facing guide.
|
||||
*
|
||||
* Coverage:
|
||||
*
|
||||
* - Below cap: silent pass.
|
||||
* - Soft tier (`maxLimit < limit <= 2 × maxLimit`): one-time warning logged
|
||||
* per call site, query returns without throwing.
|
||||
* - Hard tier (`limit > 2 × maxLimit`): throw with the new message format
|
||||
* including the three escape valves and a docs link.
|
||||
* - Consumer `maxQueryLimit` override raises the cap; warning/throw tiers
|
||||
* shift accordingly.
|
||||
* - Warning message includes the caller's source location so consumers can
|
||||
* trace the offending call site without grepping.
|
||||
*
|
||||
* @since 7.30.2
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
|
||||
import { Brainy } from '../../src/brainy'
|
||||
import { NounType } from '../../src/types/graphTypes'
|
||||
import {
|
||||
ValidationConfig,
|
||||
resetLimitWarningCache,
|
||||
validateFindParams
|
||||
} from '../../src/utils/paramValidation'
|
||||
import * as logger from '../../src/utils/logger'
|
||||
|
||||
describe('find({ limit }) two-tier enforcement (7.30.2)', () => {
|
||||
let brain: Brainy<any>
|
||||
let warnSpy: ReturnType<typeof vi.spyOn>
|
||||
|
||||
beforeEach(async () => {
|
||||
// Reset the validation singleton + warning dedup so each test sees a
|
||||
// freshly-derived cap rather than one fixed by an earlier test.
|
||||
ValidationConfig.reset()
|
||||
resetLimitWarningCache()
|
||||
// Spy directly on `prodLog.warn` — the call site the limit enforcement
|
||||
// actually uses. Spying on `console.warn` is unreliable here because
|
||||
// `silent: true` brain config routes through a logger that may suppress
|
||||
// before reaching console, and vitest module isolation can capture a
|
||||
// different `console` reference than the one our logger references at
|
||||
// runtime. The prodLog.warn entry point is what we control, so that's
|
||||
// what we observe.
|
||||
warnSpy = vi.spyOn(logger.prodLog, 'warn').mockImplementation(() => undefined)
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
if (brain) await brain.close()
|
||||
warnSpy.mockRestore()
|
||||
})
|
||||
|
||||
describe('Validator-level behavior (no brain instance needed)', () => {
|
||||
it('silent pass below cap', () => {
|
||||
const cfg = ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
expect(() => validateFindParams({ limit: cfg.maxLimit })).not.toThrow()
|
||||
expect(warnSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('soft tier warns once per call site without throwing', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
// The dedup key is `(caller, limit)`. To exercise the dedup honestly we
|
||||
// need both invocations to hit the SAME source line — extracting them
|
||||
// into a wrapper that lives at one location is the deterministic way.
|
||||
const callFromOneSite = () => validateFindParams({ limit: 1500 })
|
||||
|
||||
expect(() => callFromOneSite()).not.toThrow()
|
||||
expect(warnSpy).toHaveBeenCalledTimes(1)
|
||||
|
||||
// Same source line + same limit → dedup, no second warning
|
||||
expect(() => callFromOneSite()).not.toThrow()
|
||||
expect(warnSpy).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('warning message names the recipe + docs link', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
validateFindParams({ limit: 1500 })
|
||||
|
||||
const message = String(warnSpy.mock.calls[0][0])
|
||||
expect(message).toMatch(/find\(\{ limit: 1500 \}\)/)
|
||||
expect(message).toMatch(/exceeds the auto-configured query limit of 1000/)
|
||||
expect(message).toMatch(/new Brainy\(\{ maxQueryLimit:/)
|
||||
expect(message).toMatch(/new Brainy\(\{ reservedQueryMemory:/)
|
||||
expect(message).toMatch(/Paginate:/)
|
||||
expect(message).toMatch(/Docs: https:\/\/soulcraft\.com\/docs\/guides\/find-limits/)
|
||||
})
|
||||
|
||||
it('warning message includes the caller location from the stack', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
validateFindParams({ limit: 1500 })
|
||||
|
||||
const message = String(warnSpy.mock.calls[0][0])
|
||||
// The caller is THIS test file; the formatter strips the leading `at `
|
||||
// and emits an ` at <location>` line in the rendered message.
|
||||
expect(message).toMatch(/at .*find-limits\.test\.ts/)
|
||||
})
|
||||
|
||||
it('hard tier throws with the same message format', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
// Above 2× cap = real OOM territory = throw
|
||||
expect(() => validateFindParams({ limit: 2001 })).toThrow(
|
||||
/exceeds the auto-configured query limit of 1000/
|
||||
)
|
||||
// No warning was logged — throw fires immediately at the hard tier
|
||||
expect(warnSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('hard tier message names all three escape valves', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
try {
|
||||
validateFindParams({ limit: 5000 })
|
||||
throw new Error('expected throw')
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
expect(message).toMatch(/maxQueryLimit/)
|
||||
expect(message).toMatch(/reservedQueryMemory/)
|
||||
expect(message).toMatch(/Paginate:/)
|
||||
expect(message).toMatch(/Docs: https:\/\/soulcraft\.com\/docs\/guides\/find-limits/)
|
||||
}
|
||||
})
|
||||
|
||||
it('soft-tier warning dedup is keyed on (caller, limit) — different limits from same site fire separately', () => {
|
||||
ValidationConfig.getInstance({ maxQueryLimit: 1000 })
|
||||
|
||||
const call = (limit: number) => validateFindParams({ limit })
|
||||
|
||||
call(1500)
|
||||
call(1500)
|
||||
expect(warnSpy).toHaveBeenCalledTimes(1)
|
||||
|
||||
// Different limit value → new dedup key → second warning
|
||||
call(1800)
|
||||
expect(warnSpy).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('Consumer override via Brainy constructor', () => {
|
||||
it('maxQueryLimit raises the cap; warning/throw tiers shift accordingly', async () => {
|
||||
brain = new Brainy({
|
||||
storage: { type: 'memory' },
|
||||
silent: true,
|
||||
maxQueryLimit: 50_000
|
||||
})
|
||||
await brain.init()
|
||||
// Brainy's init path emits a one-time `prodLog.warn` for the
|
||||
// entityIdMapper system-resource notice on first-mount; clear the spy
|
||||
// history so we only observe limit-enforcement warnings below.
|
||||
warnSpy.mockClear()
|
||||
|
||||
// The old auto-derived cap would have rejected this; the override accepts it
|
||||
expect(() => validateFindParams({ limit: 10_000 })).not.toThrow()
|
||||
expect(warnSpy).not.toHaveBeenCalled()
|
||||
|
||||
// 50_000 + 1 = soft tier under the new cap → warn, not throw
|
||||
expect(() => validateFindParams({ limit: 60_000 })).not.toThrow()
|
||||
expect(warnSpy).toHaveBeenCalled()
|
||||
|
||||
// Above 2× the override (100 001) → throw
|
||||
// (Note: maxQueryLimit is hard-clamped at 100k in ValidationConfig, so the
|
||||
// effective cap is 50_000; 2× = 100_000; we cross at 100_001.)
|
||||
expect(() => validateFindParams({ limit: 100_001 })).toThrow(/exceeds/)
|
||||
})
|
||||
|
||||
it('pre-7.30.2 regression scenario: limit: 10_000 passes silently on a memory-derived cap', async () => {
|
||||
// Simulate a box where the auto-config picks a cap below 10_000 — the
|
||||
// canonical pre-7.30.2 scenario where production booking flows 500'd
|
||||
// because `validateFindParams` threw synchronously. With the
|
||||
// 25 KB-per-result calibration the cap is ~4× more generous on the
|
||||
// same hardware, but more importantly the soft tier no longer throws
|
||||
// when consumers exceed the auto-cap.
|
||||
ValidationConfig.reconfigure({ maxQueryLimit: 9000 })
|
||||
|
||||
// Pre-7.30.2 this threw. Post-7.30.2 it warns + passes.
|
||||
expect(() => validateFindParams({
|
||||
type: NounType.Event,
|
||||
where: { status: 'open' },
|
||||
limit: 10_000
|
||||
})).not.toThrow()
|
||||
expect(warnSpy).toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
|
@ -3,7 +3,7 @@
|
|||
*
|
||||
* Tests the complete fork() → listBranches() → checkout() workflow
|
||||
* to prevent regression of the v5.3.6 bug where fork() silently failed
|
||||
* to persist branches to storage (Workshop bug report).
|
||||
* to persist branches to storage (internal bug report).
|
||||
*
|
||||
* @see https://github.com/soulcraftlabs/brainy/issues/XXX
|
||||
*/
|
||||
|
|
@ -153,23 +153,23 @@ describe('Fork Persistence (v5.3.6 Bug Fix)', () => {
|
|||
expect(forkBranches).toContain(validBranch)
|
||||
})
|
||||
|
||||
it('should handle snapshot naming convention (Workshop use case)', async () => {
|
||||
// Reproduce exact Workshop snapshot workflow
|
||||
it('should handle snapshot naming convention (consumer use case)', async () => {
|
||||
// Reproduce exact Consumer snapshot workflow
|
||||
await brain.add({ data: { test: true }, type: 'concept' })
|
||||
|
||||
const commitId = await brain.commit({
|
||||
message: 'Workshop snapshot test',
|
||||
message: 'Consumer snapshot test',
|
||||
author: 'workshop@example.com'
|
||||
})
|
||||
|
||||
// Use Workshop's exact naming convention
|
||||
// Use the consumer's exact naming convention
|
||||
const timestamp = Date.now()
|
||||
const snapshotBranch = `snapshot-${timestamp}`
|
||||
|
||||
// Create snapshot branch (this is what failed in Workshop)
|
||||
// Create snapshot branch (this is what failed in the consumer report)
|
||||
await brain.fork(snapshotBranch, {
|
||||
author: 'workshop@example.com',
|
||||
message: `Snapshot: Workshop test`,
|
||||
message: `Snapshot: Consumer test`,
|
||||
metadata: {
|
||||
timestamp,
|
||||
userId: 'test-user',
|
||||
|
|
|
|||
|
|
@ -373,17 +373,17 @@ describe('getRelations() Fix (v4.1.3)', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('Comparison with Workshop Bug Report', () => {
|
||||
it('should reproduce and fix the Workshop team bug scenario', async () => {
|
||||
describe('Comparison with Internal Bug Report', () => {
|
||||
it('should reproduce and fix the a consumer team bug scenario', async () => {
|
||||
// Reproduce the exact scenario from the bug report:
|
||||
// - 524 relationships exist in GraphAdjacencyIndex
|
||||
// - brain.getRelations() was returning empty array
|
||||
|
||||
// Create entities similar to Workshop import
|
||||
// Create entities similar to Consumer import
|
||||
const entities = []
|
||||
for (let i = 0; i < 50; i++) {
|
||||
entities.push(await brain.add({
|
||||
data: `Workshop Entity ${i}`,
|
||||
data: `Consumer Entity ${i}`,
|
||||
type: NounType.Document
|
||||
}))
|
||||
}
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ describe('Memory Enhancements Integration (v5.11.0)', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('Production Workflow: Workshop Snapshot Timeline', () => {
|
||||
describe('Production Workflow: Consumer Snapshot Timeline', () => {
|
||||
it('should stream 1000 snapshots efficiently', async () => {
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@
|
|||
* Fix: centralized `resolveEntityField` helper + `BUCKETED_INDEX_FIELDS`
|
||||
* set in coreTypes.ts, used by getFieldValueForEntity.
|
||||
*
|
||||
* Reported by Muse team 2026-04-09 (handoff action BR-ORDERBY-TS).
|
||||
* Reported by a consumer 2026-04-09.
|
||||
*
|
||||
* NOTE: These tests cover FILTERED sort, which is the only supported path.
|
||||
* Unfiltered `find({ orderBy })` is explicitly rejected until the dedicated
|
||||
|
|
@ -39,7 +39,7 @@ describe('find({ orderBy }) sort bug regression', () => {
|
|||
})
|
||||
|
||||
/**
|
||||
* Muse's real use case: filtered sort of chat sessions.
|
||||
* Real consumer use case: filtered sort of chat sessions.
|
||||
* Before the fix, this returned the OLDEST entity instead of the newest
|
||||
* because getFieldValueForEntity was reading createdAt from the wrong
|
||||
* location on the entity.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
* registers on every consumer's brain. This suite is the canary that detects
|
||||
* any internal path which silently omits subtype.
|
||||
*
|
||||
* The SDK_CORE_VOCABULARY shape that surfaced Venue's `/book` 500 (2026-06-08)
|
||||
* The SDK_CORE_VOCABULARY shape from the consumer regression (2026-06-08)
|
||||
* registers `brain.requireSubtype()` rules on six NounTypes:
|
||||
* - NounType.Event
|
||||
* - NounType.Collection
|
||||
|
|
@ -17,7 +17,7 @@
|
|||
* If Brainy's own VFS / aggregation / extraction / importer / integration /
|
||||
* MCP paths skip subtype anywhere, the enforcement hook fires and Brainy itself
|
||||
* starts rejecting its own infrastructure writes. This test exercises every
|
||||
* such path under the exact shape Venue hit + brain-wide strict mode, and
|
||||
* such path under the exact shape the consumer hit + brain-wide strict mode, and
|
||||
* asserts: zero rejections.
|
||||
*
|
||||
* @since 7.30.1
|
||||
|
|
@ -32,7 +32,7 @@ describe('Brainy strict-mode self-test (7.30.1)', () => {
|
|||
|
||||
beforeEach(async () => {
|
||||
// Brain-wide strict mode ON + the exact SDK_CORE_VOCABULARY shape that
|
||||
// Venue hit. If any internal Brainy path skips subtype, the writes here
|
||||
// the consumer hit. If any internal Brainy path skips subtype, the writes here
|
||||
// will throw and fail the test.
|
||||
brain = new Brainy({
|
||||
storage: { type: 'memory' },
|
||||
|
|
|
|||
|
|
@ -67,7 +67,7 @@ describe('VFS Debug', () => {
|
|||
const rootContents = await vfs.readdir('/')
|
||||
console.log(` Root contents: ${rootContents.join(', ')}`)
|
||||
|
||||
// Try getDirectChildren (Workshop's method)
|
||||
// Try getDirectChildren (the consumer's method)
|
||||
const children = await vfs.getDirectChildren('/')
|
||||
console.log(` Direct children: ${children.length}`)
|
||||
children.forEach(child => {
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
* - stat(path, { commitId })
|
||||
* - exists(path, { commitId })
|
||||
*
|
||||
* This is the CRITICAL test for Workshop's time-travel feature.
|
||||
* This is the CRITICAL test for a consumer's time-travel feature.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest'
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
* VFS Import Verification Test
|
||||
*
|
||||
* This test verifies that brain.import() creates VFS entities correctly.
|
||||
* Created to investigate Workshop team's report of empty VFS after import.
|
||||
* Created to investigate a consumer's report of empty VFS after import.
|
||||
*
|
||||
* Expected behavior:
|
||||
* 1. Import with vfsPath creates directory entities
|
||||
|
|
@ -15,7 +15,7 @@ import { describe, it, expect, beforeEach } from 'vitest'
|
|||
import { Brainy } from '../../src/brainy.js'
|
||||
import * as XLSX from 'xlsx'
|
||||
|
||||
describe('VFS Import Verification (Workshop Bug Investigation)', () => {
|
||||
describe('VFS Import Verification (VFS import bug investigation)', () => {
|
||||
let brain: Brainy
|
||||
|
||||
beforeEach(async () => {
|
||||
|
|
@ -26,7 +26,7 @@ describe('VFS Import Verification (Workshop Bug Investigation)', () => {
|
|||
})
|
||||
|
||||
it('should create VFS entities during import with vfsPath', async () => {
|
||||
// Create test Excel file (matching Workshop scenario)
|
||||
// Create test Excel file (matching the reported scenario)
|
||||
const testData = [
|
||||
{
|
||||
'Term': 'Alice',
|
||||
|
|
@ -204,8 +204,8 @@ describe('VFS Import Verification (Workshop Bug Investigation)', () => {
|
|||
}
|
||||
}, 60000)
|
||||
|
||||
it('should match Workshop scenario exactly', async () => {
|
||||
// Replicate Workshop team's exact scenario
|
||||
it('should match the reported scenario exactly', async () => {
|
||||
// Replicate a consumer's exact scenario
|
||||
const testData = [
|
||||
{ 'Term': 'Westland', 'Definition': 'Ancient kingdom', 'Type': 'Place' },
|
||||
{ 'Term': 'Capital City', 'Definition': 'Main city', 'Type': 'Place' },
|
||||
|
|
@ -217,7 +217,7 @@ describe('VFS Import Verification (Workshop Bug Investigation)', () => {
|
|||
XLSX.utils.book_append_sheet(workbook, worksheet, 'Glossary')
|
||||
const buffer = XLSX.write(workbook, { type: 'buffer', bookType: 'xlsx' })
|
||||
|
||||
console.log('📥 Importing (Workshop scenario)...')
|
||||
console.log('📥 Importing (the reported scenario)...')
|
||||
const filename = 'Tales from Talifar Glossary.xlsx'
|
||||
const timestamp = Date.now()
|
||||
|
||||
|
|
@ -236,12 +236,12 @@ describe('VFS Import Verification (Workshop Bug Investigation)', () => {
|
|||
console.log(` - Graph edges: ${result.stats.graphEdgesCreated}`)
|
||||
console.log(` - VFS files: ${result.stats.vfsFilesCreated}`)
|
||||
|
||||
// Workshop team's check: Initialize VFS
|
||||
console.log('\n📂 Initializing VFS (Workshop fix)...')
|
||||
// Consumer's check: Initialize VFS
|
||||
console.log('\n📂 Initializing VFS (post-fix)...')
|
||||
const vfs = brain.vfs
|
||||
await vfs.init()
|
||||
|
||||
// Workshop team's check: Query root
|
||||
// a consumer's check: Query root
|
||||
console.log('🔍 Querying root directory...')
|
||||
const rootItems = await vfs.getDirectChildren('/')
|
||||
console.log(` - Items in root: ${rootItems.length}`)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Manual test to reproduce Workshop team's type filtering issue
|
||||
* Manual test to reproduce a consumer's type filtering issue
|
||||
*
|
||||
* This test verifies that brain.find({ type: NounType.Person }) actually works
|
||||
* as documented in the API.
|
||||
|
|
@ -8,7 +8,7 @@
|
|||
import { Brainy, NounType } from '../../src/index.js'
|
||||
|
||||
async function testTypeFiltering() {
|
||||
console.log('\n🔬 Testing Type Filtering Issue from Workshop Team\n')
|
||||
console.log('\n🔬 Testing Type Filtering Issue from A Consumer Team\n')
|
||||
console.log('=' .repeat(60))
|
||||
|
||||
// Create in-memory instance for testing
|
||||
|
|
@ -156,12 +156,12 @@ async function testTypeFiltering() {
|
|||
|
||||
if (passedCount === totalCount) {
|
||||
console.log('\n🎉 All tests passed! Type filtering works correctly.')
|
||||
console.log('\n💡 The Workshop team might be experiencing a different issue.')
|
||||
console.log('\n💡 The a consumer team might be experiencing a different issue.')
|
||||
console.log(' Check: storage persistence, Brainy instance reuse, or data migration')
|
||||
} else {
|
||||
console.log('\n❌ Type filtering is BROKEN in Brainy!')
|
||||
console.log('\n🐛 This is a bug that needs to be fixed.')
|
||||
console.log(' Workshop team was right - it\'s not user error.')
|
||||
console.log(' a consumer team was right - it\'s not user error.')
|
||||
}
|
||||
|
||||
console.log('\n' + '='.repeat(60) + '\n')
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
/**
|
||||
* VFS Multiple Init() Diagnostic Test
|
||||
*
|
||||
* Tests Workshop team's issue: Does calling vfs.init() multiple times
|
||||
* Tests a consumer's issue: Does calling vfs.init() multiple times
|
||||
* create duplicate root entities?
|
||||
*
|
||||
* Scenario:
|
||||
|
|
@ -72,8 +72,8 @@ describe('VFS Multiple Init Diagnostic', () => {
|
|||
expect(roots.length).toBe(1)
|
||||
})
|
||||
|
||||
it('should not create duplicate roots when creating MULTIPLE VFS instances (Workshop scenario)', async () => {
|
||||
// Simulate Workshop's scenario: Getting VFS on multiple requests
|
||||
it('should not create duplicate roots when creating MULTIPLE VFS instances (the reported scenario)', async () => {
|
||||
// Simulate the reported scenario: Getting VFS on multiple requests
|
||||
// brain.vfs returns cached instance, so this should be safe
|
||||
const vfs1 = brain.vfs
|
||||
const vfs2 = brain.vfs
|
||||
|
|
@ -107,7 +107,7 @@ describe('VFS Multiple Init Diagnostic', () => {
|
|||
|
||||
it('should handle NEW brain instances gracefully (per-user scenario)', async () => {
|
||||
// Simulate creating separate brain instances per user
|
||||
// This is closer to Workshop's getUserBrainy() pattern
|
||||
// This is closer to a consumer's getUserBrainy() pattern
|
||||
const brain2 = new Brainy({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
|
|
|
|||
|
|
@ -70,7 +70,7 @@ describe('v5.7.0 Deadlock Regression', () => {
|
|||
})
|
||||
|
||||
it('should handle imports without 12+ second delays per entity', async () => {
|
||||
// Simulate import workflow (like Workshop Excel import)
|
||||
// Simulate import workflow (like A consumer's Excel import)
|
||||
const start = Date.now()
|
||||
|
||||
// Import 5 entities with relationships
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ describe('counts.byType() fix (v6.2.2)', () => {
|
|||
})
|
||||
|
||||
it('should not return empty when entities exist (the original bug)', async () => {
|
||||
// This test reproduces the original bug from Workshop
|
||||
// This test reproduces the original bug from a consumer report
|
||||
// counts.byType({ excludeVFS: true }) returned {} even with 48 entities
|
||||
|
||||
// Arrange - Add a single entity
|
||||
|
|
@ -68,7 +68,7 @@ describe('counts.byType() fix (v6.2.2)', () => {
|
|||
})
|
||||
|
||||
it('should match counts from find() (the workaround)', async () => {
|
||||
// The Workshop workaround was to use find() and count manually
|
||||
// The consumer workaround was to use find() and count manually
|
||||
// Our fix should make both approaches return the same counts
|
||||
|
||||
// Arrange
|
||||
|
|
|
|||
|
|
@ -508,7 +508,7 @@ describe('Presets', () => {
|
|||
})
|
||||
|
||||
describe('real-world scenarios', () => {
|
||||
it('should handle Workshop glossary correctly', () => {
|
||||
it('should handle glossary correctly', () => {
|
||||
const context: ImportContext = {
|
||||
fileType: 'excel',
|
||||
rowCount: 567,
|
||||
|
|
|
|||
|
|
@ -508,8 +508,8 @@ describe('ExactMatchSignal', () => {
|
|||
})
|
||||
|
||||
describe('real-world scenarios', () => {
|
||||
it('should handle Workshop glossary import', async () => {
|
||||
// Simulate Workshop glossary with 567 terms
|
||||
it('should handle glossary import', async () => {
|
||||
// Simulate glossary with 567 terms
|
||||
const terms = [
|
||||
{ text: 'Eldoria', type: NounType.Location },
|
||||
{ text: 'Shadowfen', type: NounType.Location },
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Type Filtering Tests - Workshop Team Issue
|
||||
* Type Filtering Tests - A Consumer Team Issue
|
||||
*
|
||||
* Tests to verify that brain.find({ type: NounType.X }) correctly filters entities
|
||||
*/
|
||||
|
|
@ -7,7 +7,7 @@
|
|||
import { describe, it, expect, beforeEach } from 'vitest'
|
||||
import { Brainy, NounType } from '../../src/index.js'
|
||||
|
||||
describe('Type Filtering (Workshop Team Issue)', () => {
|
||||
describe('Type Filtering (A Consumer Team Issue)', () => {
|
||||
let brain: Brainy
|
||||
|
||||
beforeEach(async () => {
|
||||
|
|
|
|||
|
|
@ -98,9 +98,10 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
|
||||
const config = ValidationConfig.getInstance()
|
||||
|
||||
// 4GB * 0.25 = 1GB for queries
|
||||
// 1GB / 100MB = 10 units * 1000 = 10,000 limit
|
||||
expect(config.maxLimit).toBe(10000)
|
||||
// 4 GB × 0.25 = 1 GB for queries; at 25 KB / result (7.30.2 calibration)
|
||||
// → 1 GB / 25 KB = ~40_960 → floor to 40 × 1000 = 40_000.
|
||||
// Pre-7.30.2 used 100 KB / result and returned 10_000 here.
|
||||
expect(config.maxLimit).toBe(40000)
|
||||
})
|
||||
|
||||
it('should respect absolute maximum of 100k', () => {
|
||||
|
|
@ -134,12 +135,14 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
})
|
||||
|
||||
it('should respect reservedQueryMemory override', () => {
|
||||
// Reserve 1GB for queries
|
||||
// Reserve 1 GB for queries; at 25 KB / result (7.30.2) → 1 GB / 25 KB
|
||||
// = ~40_960 → floor to 40 × 1000 = 40_000. Pre-7.30.2 used 100 KB /
|
||||
// result and this returned 10_000.
|
||||
const config = ValidationConfig.getInstance({
|
||||
reservedQueryMemory: 1 * 1024 * 1024 * 1024
|
||||
})
|
||||
|
||||
expect(config.maxLimit).toBe(10000) // 1GB / 100MB * 1000
|
||||
expect(config.maxLimit).toBe(40000)
|
||||
expect(config.limitBasis).toBe('reservedMemory')
|
||||
})
|
||||
|
||||
|
|
@ -213,7 +216,7 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
it('should configure reserved memory via Brain constructor', async () => {
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'memory' },
|
||||
reservedQueryMemory: 500 * 1024 * 1024, // 500MB
|
||||
reservedQueryMemory: 500 * 1024 * 1024, // 500 MB
|
||||
silent: true
|
||||
})
|
||||
|
||||
|
|
@ -221,8 +224,9 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
|
||||
const stats = brain.getMemoryStats()
|
||||
|
||||
// 500MB / 100MB * 1000 = 5000
|
||||
expect(stats.limits.maxQueryLimit).toBe(5000)
|
||||
// 500 MB / 25 KB per result (7.30.2 calibration) = ~20_000.
|
||||
// Pre-7.30.2 used 100 KB / result and this returned 5000.
|
||||
expect(stats.limits.maxQueryLimit).toBe(20000)
|
||||
expect(stats.limits.basis).toBe('reservedMemory')
|
||||
expect(stats.config.reservedQueryMemory).toBe(500 * 1024 * 1024)
|
||||
|
||||
|
|
@ -243,8 +247,10 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
|
||||
expect(stats.memory.containerLimit).toBe(2 * 1024 * 1024 * 1024)
|
||||
expect(stats.limits.basis).toBe('containerMemory')
|
||||
// 2GB * 0.25 = 500MB -> 5000 limit
|
||||
expect(stats.limits.maxQueryLimit).toBe(5000)
|
||||
// 2 GB × 0.25 = 512 MB query budget; at 25 KB per result (7.30.2) →
|
||||
// 512 MB / 25 KB = ~20_971 → floor to 20 × 1000 = 20_000. Pre-7.30.2
|
||||
// used 100 KB per result and this returned 5_000.
|
||||
expect(stats.limits.maxQueryLimit).toBe(20000)
|
||||
|
||||
await brain.close()
|
||||
})
|
||||
|
|
@ -341,9 +347,11 @@ describe('Memory Limits - Container Detection & Smart Calculation', () => {
|
|||
|
||||
const stats = brain.getMemoryStats()
|
||||
|
||||
// Should allocate 1GB (25% of 4GB) for queries
|
||||
// Allocates 1 GB (25% of 4 GB) for queries; at 25 KB per result
|
||||
// (7.30.2 calibration) → 1 GB / 25 KB = ~40_960 → floor to 40 × 1000 =
|
||||
// 40_000. Pre-7.30.2 used 100 KB per result and this returned 10_000.
|
||||
expect(stats.memory.containerLimit).toBe(4 * 1024 * 1024 * 1024)
|
||||
expect(stats.limits.maxQueryLimit).toBe(10000)
|
||||
expect(stats.limits.maxQueryLimit).toBe(40000)
|
||||
expect(stats.limits.basis).toBe('containerMemory')
|
||||
|
||||
await brain.close()
|
||||
|
|
|
|||
|
|
@ -93,18 +93,27 @@ describe('Zero-Config Parameter Validation', () => {
|
|||
})).toThrow('invalid NounType: InvalidType')
|
||||
})
|
||||
|
||||
it('should auto-limit based on system memory', () => {
|
||||
it('should auto-limit based on system memory (two-tier enforcement)', () => {
|
||||
const config = getValidationConfig()
|
||||
|
||||
// Should reject limits above auto-configured max
|
||||
expect(() => validateFindParams({
|
||||
limit: config.maxLimit + 1
|
||||
})).toThrow(`limit exceeds auto-configured maximum of ${config.maxLimit}`)
|
||||
// 7.30.2+ design: the cap fires in two tiers.
|
||||
// - Below cap (`limit <= maxLimit`): silent pass, no signal.
|
||||
// - Soft tier (`maxLimit < limit <= 2 * maxLimit`): one-time warning
|
||||
// per call site, query proceeds. No throw.
|
||||
// - Hard tier (`limit > 2 * maxLimit`): real OOM danger zone, throw.
|
||||
|
||||
// Should accept limits at or below max
|
||||
// Below cap → pass
|
||||
expect(() => validateFindParams({ limit: config.maxLimit })).not.toThrow()
|
||||
|
||||
// Soft tier → no throw (just a warn we don't assert here — proper coverage
|
||||
// lives in the find-limits integration suite which can intercept the log).
|
||||
expect(() => validateFindParams({ limit: config.maxLimit + 1 })).not.toThrow()
|
||||
expect(() => validateFindParams({ limit: config.maxLimit * 2 })).not.toThrow()
|
||||
|
||||
// Hard tier → throw with the new message format
|
||||
expect(() => validateFindParams({
|
||||
limit: config.maxLimit
|
||||
})).not.toThrow()
|
||||
limit: config.maxLimit * 2 + 1
|
||||
})).toThrow(/exceeds the auto-configured query limit/)
|
||||
})
|
||||
|
||||
it('should auto-limit query length', () => {
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Workshop VFS Diagnostic Test
|
||||
* VFS Multi-instance Diagnostic Test
|
||||
*
|
||||
* Tests to verify VFS import behavior and identify if VFS creates only wrappers or also graph entities
|
||||
*/
|
||||
|
|
@ -7,7 +7,7 @@
|
|||
import { describe, it, expect, beforeEach } from 'vitest'
|
||||
import { Brainy, NounType } from '../../src/index.js'
|
||||
|
||||
describe('Workshop VFS Diagnostic', () => {
|
||||
describe('VFS Multi-instance Diagnostic', () => {
|
||||
let brain: Brainy
|
||||
|
||||
beforeEach(async () => {
|
||||
|
|
@ -18,7 +18,7 @@ describe('Workshop VFS Diagnostic', () => {
|
|||
})
|
||||
|
||||
it('should verify VFS creates document wrappers AND allows entity filtering', async () => {
|
||||
console.log('\n🔬 Workshop VFS Diagnostic Test\n')
|
||||
console.log('\n🔬 VFS Multi-instance Diagnostic Test\n')
|
||||
console.log('='.repeat(70))
|
||||
|
||||
// Step 1: Add entities directly (control group)
|
||||
|
|
@ -162,7 +162,7 @@ describe('Workshop VFS Diagnostic', () => {
|
|||
console.log(' - filter({ type: "person" }) → returns graph entities')
|
||||
console.log(' - filter({ type: "document" }) → returns VFS wrappers')
|
||||
console.log('')
|
||||
console.log('If Workshop gets 0 results, likely causes:')
|
||||
console.log('If a consumer gets 0 results, likely causes:')
|
||||
console.log(' ❌ Only VFS wrappers created (createEntities: false)')
|
||||
console.log(' ❌ Import not completing before query')
|
||||
console.log(' ❌ Querying different Brainy instance')
|
||||
|
|
@ -122,7 +122,7 @@ describe('VFS bulkWrite Race Condition Fix', () => {
|
|||
}
|
||||
})
|
||||
|
||||
it('should handle the Workshop template creation scenario', async () => {
|
||||
it('should handle the Consumer template creation scenario', async () => {
|
||||
// Exact scenario from bug report: template creation with mixed ops
|
||||
const operations = [
|
||||
{ type: 'mkdir' as const, path: '/project/src' },
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue