Commit graph

16 commits

Author SHA1 Message Date
ed178e2ce9 fix(ci): commit the prebuilt wasm pkg + build before test:bun (green CI on fresh clone)
The Phase-0 CI was red on all three runtimes: `src/embeddings/wasm/pkg/` (the
wasm-pack output the dynamic `import('./pkg/candle_embeddings.js')` resolves) was
gitignored + untracked, so tsc failed to resolve it on a fresh clone (TS2307), and
the Bun job ran test:bun without building dist first.

Fixes:
- Track the 3.1 MB prebuilt pkg (candle_embeddings.js/.d.ts + _bg.wasm/.d.ts). It
  ships in the npm tarball anyway; versioning it lets consumers + CI build without
  a Rust/wasm-pack toolchain and makes the shipped artifact reproducible (closes
  the "publish ships whatever the maintainer last built" supply-chain gap). The
  top-level .gitignore intended this (`!*.wasm` etc.) but excluded the whole dir,
  so the re-includes were dead, and a nested wasm-pack `.gitignore *` also masked
  it — force-added past both.
- CI Bun job: `npm run build` before `test:bun` (it imports the built dist/).

Verified against a tracked-files-only tree (fresh-clone sim): typecheck 0, build 0,
test:bun 8/8.
2026-07-01 14:40:11 -07:00
a8fcc3dfbc chore: gitignore Claude Code harness scheduled-tasks lockfile 2026-05-15 11:26:11 -07:00
089a4d4141 docs: add Claude Code project guide and verified architecture reference
Contributor-friendly CLAUDE.md with setup, conventions, and architecture
overview. Comprehensive skills/architecture.md verified against actual
codebase covering all 31 source directories, 38+ exports, and major
subsystems (distributed, transactions, neural, CLI, MCP, COW, etc).

Remove CLAUDE.md from .gitignore since the new version is designed
for public contributors.
2026-02-10 09:09:18 -08:00
39b099cafc feat: add migration system with error handling, validation, and enterprise hardening
- MigrationRunner: per-entity error tracking (non-fatal), maxErrors bail-out,
  static validateMigrations() called in constructor, branch error propagation
- RefManager: updateRefMetadata() method for clean metadata updates
- brainy.ts: eliminate as-any casts in migration methods, use updateRefMetadata,
  forward maxErrors through full chain including branch migrations
- Types: MigrationError interface, errors field on MigrationResult, maxErrors on MigrateOptions
- Package exports: MigrationError type, migrate() on BrainyInterface, autoMigrate config
- 31 integration tests covering error handling, validation, branch error propagation
- Documentation: docs/guides/schema-migrations.md
2026-02-09 16:13:14 -08:00
da7d2ed29d feat: migrate embeddings to Candle WASM + remove semantic type inference
Major architectural changes:

1. EMBEDDINGS ENGINE (ONNX → Candle WASM):
   - Replace ONNX Runtime with Rust Candle compiled to WASM
   - Embedded model in WASM binary (no external downloads)
   - Quantized Q8 precision with <50MB memory footprint
   - Zero-download, offline-first operation
   - Same embedding quality (all-MiniLM-L6-v2)

2. REMOVE SEMANTIC TYPE INFERENCE:
   - Delete embeddedKeywordEmbeddings.ts (14MB of pre-computed embeddings)
   - Remove typeAwareQueryPlanner.ts and semanticTypeInference.ts
   - Remove VerbExactMatchSignal (uses keyword embeddings)
   - Update SmartRelationshipExtractor to 3 signals (55%/30%/15% weights)

API CHANGES (requires v7.0.0):
- Removed: inferTypes(), inferNouns(), inferVerbs(), inferIntent()
- Removed: getSemanticTypeInference(), SemanticTypeInference class
- Removed: TypeInference, SemanticTypeInferenceOptions types

Users can still use natural language queries in find() - they just
need to specify type explicitly for type-optimized searches.

PACKAGE SIZE IMPACT:
- Compressed: 90.1 MB → 86.2 MB (-4.3%)
- Uncompressed: 114.4 MB → 100.3 MB (-12%)
- ~448K lines of code removed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:52:34 -08:00
1f59aa2013 feat: replace transformers.js with direct ONNX WASM for Bun compatibility
- Remove @huggingface/transformers dependency (539MB native binaries)
- Add direct ONNX Runtime Web embedding engine
- Bundle all-MiniLM-L6-v2-q8 model (24MB, no runtime downloads)
- Works with Node.js, Bun, and bun build --compile
- Air-gap compatible: fully self-contained, no internet required

New WASM embedding components:
- WASMEmbeddingEngine: Main integration class
- WordPieceTokenizer: Pure TypeScript tokenizer
- EmbeddingPostProcessor: Mean pooling + L2 normalization
- ONNXInferenceEngine: Direct ONNX Runtime Web wrapper
- AssetLoader: Model file loading

Tests added:
- 11 WASM embedding integration tests
- 8 Bun compatibility tests

New npm scripts:
- test:wasm - Run WASM embedding tests
- test:bun - Run tests with Bun
- test:bun:compile - Build and run compiled binary
2025-12-17 17:42:37 -08:00
759e7fabd0 docs: add production service architecture guide to public docs
Added comprehensive production service architecture guide with singleton patterns, caching strategies, and performance optimization for Express/Node.js services.

Changes:
- NEW: docs/PRODUCTION_SERVICE_ARCHITECTURE.md - Complete guide for using Brainy in production services
- CHANGED: .gitignore - Removed PRODUCTION_*.md pattern to allow public documentation
- CHANGED: README.md - Added subtle link to production architecture guide in "Production MVP" section

Guide covers:
- Instance-per-request anti-pattern (40x memory waste)
- Singleton pattern implementation (40x memory reduction, 30x faster)
- Three implementation patterns (simple, service class, middleware)
- Optimization strategies (cache sizing, lazy loading, warm-up)
- Concurrency and thread safety
- Production checklist and monitoring

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 16:21:27 -08:00
04477fef84 refactor: replace Brainy with TransformerEmbedding in buildEmbeddedPatterns
- Replace Brainy with TransformerEmbedding for embedding patterns
- Update initialization logic to support TransformerEmbedding features like `localFilesOnly`
- Adjust embedding logic to use TransformerEmbedding's embed method
- Remove unused Brainy-related code (e.g., close method)
2025-09-29 10:05:46 -07:00
6113b39109 chore: add .brainy runtime folder to gitignore 2025-09-22 16:01:31 -07:00
ed64c266ec feat: add distributed architecture with sharding and coordination
- Wire up distributed components (Coordinator, ShardManager, CacheSync)
- Implement automatic sharding for S3 storage (256 shards)
- Add read/write separation for operational modes
- Zero-config automatic detection for distributed mode
- Add mutex implementation for thread safety
- Fix metadata filtering in find operations
- Fix neural API vector similarity calculations
- Improve batch operations performance
- Add Bluesky distributed setup example

BREAKING CHANGE: None - backward compatible
2025-09-22 15:45:35 -07:00
0996c72468 feat: Brainy 3.0 - Production-ready Triple Intelligence database
Major improvements and simplifications:
- Simplified to Q8-only model precision (99% accuracy, 75% smaller)
- Removed WAL augmentation (not needed with modern filesystems)
- Eliminated all fake/stub code - 100% production-ready
- Added comprehensive cloud deployment support (Docker, K8s, AWS, GCP)
- Enhanced distributed system capabilities
- Improved Triple Intelligence find() implementation
- Added streaming pipeline for large-scale operations
- Comprehensive test coverage with new test suites

Breaking changes:
- Renamed BrainyData to Brainy (simpler, cleaner)
- Removed FP32 model option (Q8 provides 99% accuracy)
- Removed deprecated augmentations

Performance improvements:
- 10x faster initialization with Q8-only
- Reduced memory footprint by 75%
- Better scaling for millions of items

Co-Authored-By: Recovery checkpoint system
2025-09-11 16:23:32 -07:00
d7d2d749b6 chore: move internal planning documents to private .strategy folder
- Moved strategy and planning documents out of repository
- Added .strategy/ to .gitignore for private documents
- These files will be removed from git history in next step
2025-08-30 08:53:51 -07:00
0c0d929174 chore: add plan.md and CLAUDE.md to gitignore
Prevent planning and instruction files from being committed to the repository.
These files are for development workflow only and should not be tracked.
2025-08-28 15:08:09 -07:00
df81c786e4 chore: Add .npmignore to exclude models from npm package
- npm package will be ~5.5MB instead of 200MB+
- Models download automatically on first use
- Added models-cache/ to .gitignore
2025-08-26 13:37:44 -07:00
ef1745af8f Fix: Update all search calls to 2-parameter API 2025-08-26 12:49:31 -07:00
9c87982a7d 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™
MAJOR RELEASE: Complete evolution of Brainy with groundbreaking features and performance.

🎯 KEY FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Triple Intelligence™ Engine
  - Unified Vector + Metadata + Graph search
  - O(log n) performance on all operations
  - 3ms average search latency at any scale

 API Consolidation
  - 15+ search methods → 2 clean APIs
  - search() for vector similarity
  - find() for natural language queries

 Natural Language Processing
  - 220+ pre-computed NLP patterns
  - Instant context understanding
  - "Show me recent React components with tests"

 Zero Configuration
  - Works instantly, no setup required
  - Built-in embedding models (no API keys)
  - Smart defaults for everything
  - Automatic optimization

 Enterprise Features (Free for Everyone)
  - Scales to 10M+ items
  - Write-Ahead Logging (WAL) for durability
  - Distributed architecture with sharding
  - Read/write separation
  - Connection pooling & request deduplication
  - Built-in monitoring & health checks

 Universal Compatibility
  - Node.js, Browser, Edge Workers
  - 4 Storage Adapters (Memory, FileSystem, OPFS, S3)
  - TypeScript with full type safety
  - Worker-based embeddings

📦 WHAT'S INCLUDED:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Core AI Database with HNSW indexing
• 19 Production-ready augmentations
• Universal Memory Manager
• Complete CLI with all commands
• Brain Cloud integration (soulcraft.com)
• Comprehensive documentation
• 52 test files with 400+ tests
• Migration guide from 1.x

📊 PERFORMANCE:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Initialize: 450ms (24MB memory)
• Search: 3ms average (up to 10M items)
• Metadata Filter: 0.8ms (O(log n))
• Bulk Import: 2.3s per 1000 items
• Production Scale: 5.8ms at 10M items

🔧 TECHNICAL IMPROVEMENTS:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• TypeScript compilation: 153 errors → 0
• Memory usage: 200MB → 24MB baseline
• Circular dependencies resolved
• Worker thread communication fixed
• Storage adapter consistency
• Request coalescing for 3x performance

🛠️ CLI FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• brainy add - Smart data ingestion
• brainy find - Natural language search
• brainy search - Vector similarity
• brainy chat - AI conversation mode
• brainy cloud - Brain Cloud integration
• brainy augment - Manage extensions
• 100% API compatibility

📚 DOCUMENTATION:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Professional README with examples
• Quick Start guide (5 minutes)
• Enterprise Features guide
• Migration guide from 1.x
• API reference
• Architecture documentation

🌟 USE CASES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• AI memory layer for chatbots
• Semantic document search
• Code intelligence platforms
• Knowledge management systems
• Real-time recommendation engines
• Customer support automation

MIT License - Enterprise features included free for everyone.
No premium tiers, no paywalls, no limits.

Built with ❤️ by the Brainy community.
Visit https://soulcraft.com for Brain Cloud integration.
2025-08-26 12:32:21 -07:00