Commit graph

6 commits

Author SHA1 Message Date
1f7e365a4e chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
970e08c466 feat(8.0): reserved-field contract — one canonical location, typed prevention, unified read/write
Brainy-owned field names (noun/verb, subtype, createdAt, updatedAt,
confidence, weight, service, data, createdBy, _rev) now have exactly one
home — top level — enforced by three layers driven from a single source of
truth, src/types/reservedFields.ts (RESERVED_ENTITY_FIELDS /
RESERVED_RELATION_FIELDS, exported):

1. Compile time — AddParams/UpdateParams/RelateParams/UpdateRelationParams
   metadata (and the transact() ops that extend them) reject a literal
   reserved key as a TypeScript error while keeping generic T ergonomics
   (typed bags, untyped brains, index-signature shapes, and a documented
   exemption for T-declared reserved keys). Pinned by @ts-expect-error
   type tests run under vitest typecheck mode on every unit run.

2. Write time — the 7.x update() remap is ported to 8.0 and extended to
   every write path: add/update/relate/updateRelation, their transact()
   mirrors, and db.with() overlays. User-settable fields lift to their
   dedicated param (top-level wins when both are supplied — closes the 7.x
   trap where update({metadata:{confidence}}) silently no-oped), and
   system-managed fields drop with a one-shot warning naming the right
   path. A remapped subtype satisfies subtype-pairing enforcement exactly
   like a top-level one.

3. Read time — every storage combine goes through one canonical hydration
   helper (hydrateNounWithMetadata / hydrateVerbWithMetadata over
   splitNoun/VerbMetadataRecord), so reserved fields surface ONLY top-level
   and entity/relation.metadata carry ONLY custom fields on live reads,
   batch reads, paginated listings, getRelations by source/target, streamed
   verbs, and historical asOf() materialization alike.

Read-path echoes found and fixed (previously the full stored record —
including the verb type key — leaked inside metadata): noun pagination,
verb pagination, getVerbsBySource/ByTarget (adjacency + shard fallback),
getVerbsBySourceBatch (which also dropped subtype/data), and the
filesystem verb stream. getRelations() results now surface
confidence/updatedAt top-level via verbsToRelations, updateRelation() no
longer erases service/createdBy, relate() persists its top-level
confidence/service params, and the dead convertHNSWVerbToGraphVerb echo
path is deleted. Import paths (CLI extract, deduplicator, coordinators,
neural import) write confidence through the dedicated param instead of the
bag. UpdateRelationParams is now exported from the package root.

Documented for consumers in docs/concepts/consistency-model.md ("Reserved
fields") and RELEASES.md. Regression tests ported from the 7.x fix and
extended to the full 8.0 contract (17 runtime tests + 41 type-level
assertions); full unit suite 1427/1427, db-mvcc integration 24/24.
2026-06-11 13:13:09 -07:00
cd875294ad fix: eliminate flaky test timeouts and add storage adapters guide
- Switch vitest pool from threads to forks for process isolation
- Disable v8 coverage by default (causes vitest worker RPC timeout)
- Increase hookTimeout 30s to 60s for MetadataIndexManager init under load
- Reduce O(1) space test entity count, replace brittle timing assertions
- Add docs/guides/storage-adapters.md with verified batch config values
2026-01-31 09:11:20 -08:00
42ae5be455 feat: ID-first storage architecture + remove memory-unsafe APIs (v6.0.0)
BREAKING CHANGES:

**ID-First Storage Paths**
- Direct O(1) entity access without type lookups
- Before: entities/nouns/{TYPE}/metadata/{SHARD}/{ID}.json
- After:  entities/nouns/{SHARD}/{ID}/metadata.json
- Migration handled automatically on first init()

**Removed Memory-Unsafe APIs**
- Removed brain.merge() - loaded all entities into memory
- Removed brain.diff() - loaded all entities into memory
- Removed brain.data().backup() - loaded all entities into memory
- Removed brain.data().restore() - depended on backup()
- Removed CLI commands: backup, restore, cow merge

**Migration Paths**
- merge() → Use checkout() or manually copy entities with pagination
- diff() → Use asOf() with manual paginated comparison
- backup() → Use fork() for instant COW snapshots
- restore() → Use checkout() to switch to snapshot branch

Core Improvements:
-  All 8 storage adapters properly call super.init()
-  GraphAdjacencyIndex integration in BaseStorage.init()
-  Fixed ID-first path bugs (vector.json → vectors.json)
-  Fixed MemoryStorage.initializeCounts() for ID-first paths
-  New VFS APIs: du(), access(), find()
-  Comprehensive documentation with migration guides

Storage Adapters Fixed:
- MemoryStorage, FileSystemStorage, AzureBlobStorage
- GCSStorage, R2Storage, S3CompatibleStorage
- OPFSStorage, HistoricalStorageAdapter

Files Changed: 28 files, +1,075/-1,933 lines (net -858)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 16:46:11 -08:00
93d2d70a44 fix: resolve VFS tree corruption from blob errors (v5.8.0)
CRITICAL FIX: Single blob read error no longer corrupts entire VFS tree

Root Causes Fixed:
1. Uncaught blob errors in readFile() triggered VFS re-initialization
2. Race conditions in initializeRoot() created duplicate roots
3. Wrong root selection algorithm (oldest vs most children)

Architectural Solution:
- **Error Isolation**: Blob errors caught and re-thrown as VFSError
  VFS tree structure completely isolated from file content errors
  (src/vfs/VirtualFileSystem.ts:288-321)

- **Singleton Promise Pattern**: Prevents duplicate root creation
  Concurrent init() calls wait for same initialization promise
  Acts as automatic mutex without custom lock class
  (src/vfs/VirtualFileSystem.ts:180-199)

- **Smart Root Selection**: Selects root with MOST children (not oldest)
  Auto-heals existing duplicates on init()
  Logs cleanup suggestions for empty roots
  (src/vfs/VirtualFileSystem.ts:283-334)

Production Impact:
- Workshop production: 5 duplicate roots, 1,836 files orphaned
- After fix: Zero duplicate roots possible, auto-healing
- All 100 VFS tests pass 

Additional Fix: Remove Sharp native dependency (v5.8.0)

ImageHandler rewritten using pure JavaScript:
- exifr (already installed) for EXIF extraction
- probe-image-size for image dimensions/format
- Zero native dependencies (removed 10MB of native binaries)
- All 25 image handler tests pass 
- No more test crashes from Sharp/libvips worker thread issues

Test Results:
- VFS tests: 100/100 pass 
- Image handler tests: 25/25 pass 
- Overall: 1157/1200 tests pass (18 pre-existing timeout issues)
- Build: Successful, zero TypeScript errors 

Features Complete (TIER 1 - v5.8.0):
- Transaction system (36 unit + 35 integration tests)
- Duplicate check optimization (O(n) → O(log n))
- GraphIndex pagination
- Comprehensive filter documentation
2025-11-14 11:27:35 -08:00
9c87982a7d 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™
MAJOR RELEASE: Complete evolution of Brainy with groundbreaking features and performance.

🎯 KEY FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Triple Intelligence™ Engine
  - Unified Vector + Metadata + Graph search
  - O(log n) performance on all operations
  - 3ms average search latency at any scale

 API Consolidation
  - 15+ search methods → 2 clean APIs
  - search() for vector similarity
  - find() for natural language queries

 Natural Language Processing
  - 220+ pre-computed NLP patterns
  - Instant context understanding
  - "Show me recent React components with tests"

 Zero Configuration
  - Works instantly, no setup required
  - Built-in embedding models (no API keys)
  - Smart defaults for everything
  - Automatic optimization

 Enterprise Features (Free for Everyone)
  - Scales to 10M+ items
  - Write-Ahead Logging (WAL) for durability
  - Distributed architecture with sharding
  - Read/write separation
  - Connection pooling & request deduplication
  - Built-in monitoring & health checks

 Universal Compatibility
  - Node.js, Browser, Edge Workers
  - 4 Storage Adapters (Memory, FileSystem, OPFS, S3)
  - TypeScript with full type safety
  - Worker-based embeddings

📦 WHAT'S INCLUDED:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Core AI Database with HNSW indexing
• 19 Production-ready augmentations
• Universal Memory Manager
• Complete CLI with all commands
• Brain Cloud integration (soulcraft.com)
• Comprehensive documentation
• 52 test files with 400+ tests
• Migration guide from 1.x

📊 PERFORMANCE:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Initialize: 450ms (24MB memory)
• Search: 3ms average (up to 10M items)
• Metadata Filter: 0.8ms (O(log n))
• Bulk Import: 2.3s per 1000 items
• Production Scale: 5.8ms at 10M items

🔧 TECHNICAL IMPROVEMENTS:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• TypeScript compilation: 153 errors → 0
• Memory usage: 200MB → 24MB baseline
• Circular dependencies resolved
• Worker thread communication fixed
• Storage adapter consistency
• Request coalescing for 3x performance

🛠️ CLI FEATURES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• brainy add - Smart data ingestion
• brainy find - Natural language search
• brainy search - Vector similarity
• brainy chat - AI conversation mode
• brainy cloud - Brain Cloud integration
• brainy augment - Manage extensions
• 100% API compatibility

📚 DOCUMENTATION:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• Professional README with examples
• Quick Start guide (5 minutes)
• Enterprise Features guide
• Migration guide from 1.x
• API reference
• Architecture documentation

🌟 USE CASES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• AI memory layer for chatbots
• Semantic document search
• Code intelligence platforms
• Knowledge management systems
• Real-time recommendation engines
• Customer support automation

MIT License - Enterprise features included free for everyone.
No premium tiers, no paywalls, no limits.

Built with ❤️ by the Brainy community.
Visit https://soulcraft.com for Brain Cloud integration.
2025-08-26 12:32:21 -07:00