Brainy 7.30.0 introduced a memory-derived synchronous cap on `find({ limit })`
to prevent OOM. The cap was sound in intent but ~4x too conservative in
calibration: assumed 100 KB per result while typical entity footprint is 7-10 KB
(384-dim float32 vector ≈ 1.5 KB + standard fields + metadata). On a 900 MB
free-memory box the cap derived to 9000 — breaking common safety-cap patterns
like `find({ type, where, limit: 10_000 })` that typically return 10-500
entities. Surfaced as a runtime regression with cascading 500s degrading
production dashboards.
Three concurrent fixes:
A. RECALIBRATE THE FORMULA
- src/utils/paramValidation.ts:175,196,212 — the three memory-derived priorities
(reservedQueryMemory / containerMemory / freeMemory) all divided by
100 * 1024 * 1024 (100 KB per result, ~10-15x over conservative). Replaced
with a new MAX_LIMIT_KB_PER_RESULT = 25 constant that matches observed
entity size.
- Result: 4 GB container cap goes 10_000 → 40_000; 2 GB cap goes 5_000 →
20_000; 900 MB free-memory cap goes 9_000 → ~36_000. 100k hard ceiling
unchanged. `maxQueryLimit` / `reservedQueryMemory` constructor overrides
unchanged in behavior.
B. TWO-TIER ENFORCEMENT (warn-then-throw)
- Below cap (limit <= maxLimit): silent pass, unchanged.
- Soft tier (maxLimit < limit <= 2 * maxLimit): NEW — one-time warning per
call site (dedup keyed on caller stack frame + limit value), query
proceeds. Pre-7.30.2 code that relied on the cap silently allowing typical
safety-cap limits keeps working; the warning teaches the recipe so consumers
can fix it intentionally.
- Hard tier (limit > 2 * maxLimit): throw with the same teaching message
format. Real OOM territory; the cap stops being a recommendation and becomes
a guardrail.
- The 2x soft margin absorbs typical safety-cap patterns (limit: 10_000
against a 9 K-cap box) without disabling OOM protection. Real OOM territory
on a JS in-memory brain is hundreds of thousands of results, not 10x the
safety cap.
C. IMPROVED ERROR / WARNING MESSAGE
- Same shape as the 7.30.1 enforcement-error messages: state the problem,
name the three escape valves (maxQueryLimit / reservedQueryMemory /
pagination), include caller location, link to docs.
- Extracted findCallerLocation() helper from brainy.ts to a new
src/utils/callerLocation.ts so both the subtype enforcement (7.30.1) and
the limit enforcement (7.30.2) share one implementation without circular
imports.
DOCS
- New docs/guides/find-limits.md (public: true) — full reference: why the cap
exists, the four memory sources the auto-config considers, the three escape
valves with when-to-use-which guidance, and an explicit "pagination is the
future-proof pattern" callout (8.0 may tighten the cap further; pagination
keeps working unchanged).
- docs/api/README.md find() entry gets a one-paragraph `limit` tip + pointer
to the new guide.
- RELEASES.md v7.30.2 entry.
TESTS
- New tests/integration/find-limits.test.ts (9 tests): below-cap silent pass;
soft-tier warns once per call site (dedup verified by exercising same vs.
different source lines via wrapper closures); soft-tier message format
(names all three escape valves + docs link); soft-tier message includes
caller location; hard-tier throws; hard-tier message format same as
soft-tier; consumer maxQueryLimit override raises the cap and shifts both
tiers accordingly; pre-7.30.2 regression scenario explicitly covered.
- tests/unit/utils/memoryLimits.test.ts — 4 tests updated for the recalibrated
cap values (hardcoded expected numbers bumped 4x to match new 25 KB/result
assumption).
- tests/unit/utils/paramValidation.test.ts — auto-limit test extended to cover
the three-tier semantics (below-cap pass / soft-tier silent / hard-tier
throw).
- Existing suites unchanged: subtype-and-facets 26/26, verb-subtype-and-
enforcement 30/30, strict-mode-self-test 13/13. Unit 1468/1468.
CORTEX COMPATIBILITY
- Zero Cortex changes required. Every change is JS-side: formula recalibration
runs in ValidationConfig.constructor(), two-tier enforcement runs in
validateFindParams(), both fire before any storage / index / Cortex call.
- The new guide notes that Brainy 8.0's Datomic-style Db.find() may tighten
per-call limits to keep snapshot semantics cheap; pagination remains the
pattern that's guaranteed to keep working.
REPO-WIDE CLEANUP
Brainy is the only Soulcraft project that is open source. This commit also
scrubs closed-source product names and product-specific class/field references
from every tracked file in the repo (src/, docs/, tests/, RELEASES.md,
CHANGELOG.md). Consumer-reported bugs, regression scenarios, and release
notes now refer to "a consumer", "a downstream application", "a production
deployment", or "an internal report" — never to the named product. Two
product-named test files renamed to neutral diagnostic names. CLAUDE.md gains
a project-level guard rule documenting the policy and an example list of the
identifiers that may not appear in tracked code.
Verification
- npx tsc --noEmit: clean
- npm test: 1468 / 1468 unit
- All four integration subtype + verb + strict + find-limits suites: 78/78
- npm run build: clean
- Closed-source product reference audit: clean
8.4 KiB
Brainy - Claude Code Project Guide
This file provides guidance for Claude Code (and human contributors) when working on the Brainy codebase.
Cross-Project Coordination
Handoff file: /home/dpsifr/.strategy/PLATFORM-HANDOFF.md
At session START: Read the handoff. Find rows where Owner = Brainy. Act on those first.
At session END: Mark completed actions ✅, delete rows you finished, delete threads with zero remaining actions. File must not grow. If you shipped anything consumers need to know about, update RELEASES.md before closing.
Brainy's current open actions: None. MIT open-source — no platform-specific actions.
Current version: @soulcraft/brainy@7.19.10
Project Overview
Brainy is a Universal Knowledge Protocol -- a Triple Intelligence database that combines vector similarity search, graph traversal, and metadata filtering into a single TypeScript library. Published as @soulcraft/brainy on npm under the MIT license.
Getting Started
npm install # Install dependencies
npm run build # Build the project
npm test # Run test suite (Vitest)
Architecture
Full architecture reference: .claude/skills/architecture.md
Core Systems
- Storage (
src/storage/): Pluggable storage backends via StorageAdapter interface (src/coreTypes.ts) - Vector Search (
src/hnsw/): HNSW approximate nearest neighbor search - Graph Engine (
src/graph/): Relationship traversal with adjacency index and pathfinding - Metadata Index (
src/utils/metadataIndex.ts): O(1) exact match, O(log n) range queries - Triple Intelligence (
src/triple/): Unified query combining all three intelligence types - Aggregation Engine (
src/aggregation/): Write-time incremental SUM/COUNT/AVG/MIN/MAX with GROUP BY and time windows - Virtual Filesystem (
src/vfs/): Full VFS with semantic search
Type System
- NounType (42 types): Entity classification -- Person, Concept, Collection, Document, Task, etc.
- VerbType (127 types): Relationship types -- Contains, RelatedTo, PartOf, Creates, DependsOn, etc.
- Defined in
src/types/graphTypes.ts
Code Standards
TypeScript
- Strict mode enabled
- Target: ES2020, NodeNext module resolution
- All new code must be TypeScript
- Follow existing patterns -- read related code before writing
Quality
- All code must compile without errors
- All code must have working tests that exercise real behavior
- No stub returns (
return {} as any) - No incomplete implementations with TODO comments
- If something can't be fully implemented, throw an explicit error rather than faking it
Verification Before Code Changes
- Check that interfaces and methods actually exist before using them
- Check that type properties are in the type definitions
- Run
npm test-- tests must pass - Run
npm run build-- build must succeed
Testing
- Framework: Vitest
- Tests in
tests/(unit, integration, benchmarks, comprehensive) - Use in-memory storage for speed where possible
- Tests must exercise real behavior, not mock it
- Benchmarks are in
tests/benchmarks/(not tests/performance/)
Commit Conventions
Use Conventional Commits:
feat: add new feature (minor version bump)
fix: resolve bug (patch version bump)
docs: update documentation (patch version bump)
perf: improve performance (patch version bump)
refactor: restructure code (patch version bump)
test: add/update tests (patch version bump)
Important: Never use BREAKING CHANGE in commit messages. Major version bumps are manual decisions only (npm run release:major).
Docs Pipeline — soulcraft.com/docs
Docs in docs/**/*.md are published with the npm package (included in files) and synced to soulcraft.com/docs on every portal deploy. Frontmatter controls what appears publicly.
Docs check triggers
Run the docs check whenever the user says ANY of:
- "commit, publish, release" / "release" / "publish"
- "update the docs" / "make sure docs are accurate" / "check the docs"
- "review docs" / "clean up docs"
Pre-release docs check (MANDATORY before every release)
When the user says "commit, publish, release" or any variation, before committing:
- Scan all files changed in this session (and any recently added
docs/*.mdfiles) - For each changed/new doc, decide: is this useful to external users?
- Yes → ensure it has complete frontmatter (add or update it)
- No (internal, migration, dev-only) → ensure it has no frontmatter or
public: false
- For docs that already have frontmatter, verify:
descriptionstill matches the actual contentnextlinks still exist and are still the right follow-up pagestitlematches the doc's h1
- Include frontmatter changes in the commit
Frontmatter format
---
title: Human-readable title
slug: category/page-name # URL: soulcraft.com/docs/category/page-name
public: true # false or absent = not published
category: getting-started | concepts | guides | api
template: guide | concept | api # controls layout on soulcraft.com
order: 1 # sidebar position within category (lower = first)
description: One sentence. What this doc covers and why it matters.
next: # "Next steps" links shown at bottom of page
- category/other-slug
---
Category guide
| category | use for |
|---|---|
getting-started |
installation, quick start, first steps |
concepts |
how the system works, mental models |
guides |
how to do specific things, recipes |
api |
method reference, signatures, parameters |
What stays internal (no frontmatter / public: false)
- Release guides, developer learning paths
- Migration guides for old versions (v3→v4, v5.11)
- Architecture analysis docs (clustering algorithms, etc.)
- Anything in
docs/internal/ - Deployment/ops/cost docs (cloud-run, kubernetes, cost-optimization)
Release Process
Fully automated via scripts/release.sh:
npm run release:dry # Preview (no changes)
npm run release:patch # Bug fixes
npm run release:minor # New features
npm run release:major # Breaking changes (rare, manual decision)
The script: verifies clean git state, builds, tests, bumps version, updates CHANGELOG.md, commits, tags, pushes, publishes to npm, and creates a GitHub release.
After a successful release, remind the user:
"Published. Deploy portal to pick up the new docs → go to the portal project and deploy."
Do NOT deploy portal from here. Portal is always deployed separately from within the portal project.
Closed-Source Product Names — HARD RULE
Brainy is the only Soulcraft open-source project. Nothing in this repo — code, JSDoc, tests,
docs, RELEASES.md, CHANGELOG.md, commit messages — may reference closed-source Soulcraft
products by name (Workshop, Venue, Memory, Muse, Hall, Forge, Academy, Pulse, Heart,
Collective, SDK) or by their specific class/method names (BookingDraftService,
getDemandHeatmap, systemKind, etc.).
When recording a consumer-reported bug, regression scenario, or release note:
- Refer to "a consumer", "a downstream application", "a production deployment", or "an internal report" — never name the product.
- All doc examples must use generic domain values (
'employee','customer','invoice','milestone',OrderService,/orders/...), not product-specific schemas. - Internal session artifacts (
.strategy/,~/.claude/plans/, handoff files outside the repo) MAY name products — those are not public.
If you catch yourself typing a product name into a tracked file, stop and rephrase.
Performance Claims
When documenting performance characteristics:
- MEASURED: Cite the test file and line number
- PROJECTED: Clearly label as extrapolated from tested scale
- Never claim a performance figure without context or evidence
Debugging
When a bug persists through 2+ fix attempts, switch to systematic debugging:
- Add comprehensive logging at every step
- Test with production-like data
- Trace the complete execution path
- Check both library code and consumer code
- Verify with actual test execution before declaring fixed
Key Paths
- Main class:
src/brainy.ts - Public API:
src/index.ts(38+ exports) - Storage interface:
src/coreTypes.ts - Type definitions:
src/types/ - Strategy/planning docs:
.strategy/(gitignored, not public)