fae88d0fdd
chore(release): 7.19.5
2026-02-19 17:06:33 -08:00
b98bd532d2
chore(release): 7.19.4
2026-02-19 17:05:34 -08:00
33ea5e322a
chore(release): 7.19.3
2026-02-19 17:04:37 -08:00
9d5a74abe1
chore(release): 7.19.2
2026-02-18 15:38:59 -08:00
4602960e86
chore(release): 7.18.0
2026-02-16 16:58:31 -08:00
d453539a4d
chore(release): 7.17.0
2026-02-09 16:13:52 -08:00
e9f6a1b461
chore(release): 7.16.0
2026-02-09 12:08:34 -08:00
edb5ec4696
chore(release): 7.15.5
2026-02-02 09:29:53 -08:00
a88268fd25
chore(release): 7.15.4
2026-02-02 09:21:31 -08:00
0ec3d85c39
chore(release): 7.15.3
2026-02-02 08:52:53 -08:00
389e9d6258
chore(release): 7.15.2
2026-02-01 17:56:14 -08:00
b87426409d
chore(release): 7.15.0
2026-02-01 13:05:10 -08:00
3a21bf62b7
chore(release): 7.14.0
2026-02-01 11:12:35 -08:00
0292cf2ddf
chore(release): 7.13.0
2026-02-01 10:49:36 -08:00
ac7a1f772c
chore(release): 7.12.0
2026-02-01 08:25:50 -08:00
b0439fbc26
chore(release): 7.11.0
2026-01-31 12:43:40 -08:00
e384afcdac
chore(release): 7.10.0
2026-01-31 12:02:59 -08:00
88d0c89143
chore(release): 7.9.3
2026-01-28 08:52:52 -08:00
1ffdfa70ae
chore(release): 7.9.1
2026-01-27 15:40:46 -08:00
32dbdcec61
chore(release): 7.9.0
2026-01-27 13:48:43 -08:00
6c27f9f7fd
chore(release): 7.8.0
2026-01-27 10:30:38 -08:00
cca1cd8ce2
feat: add structured content extraction and batch embedding optimization to highlight()
...
Fix highlight() hanging on structured text input by addressing 3 root causes:
1. embedBatch() now uses native WASM batch API (single forward pass instead
of N individual embed() calls via Promise.all)
2. highlight() auto-detects content type (plain text, rich-text JSON, HTML,
Markdown) and extracts meaningful text segments. Supports TipTap, Slate.js,
Lexical, Draft.js, and Quill Delta formats. New contentType hint and
contentExtractor callback for custom parsers.
3. Semantic matching phase has 10s timeout - falls back to text-only matches
instead of hanging indefinitely.
Also fixes extractTextContent() array check: uses type-based detection
(typeof data[0] === 'number') instead of length-based (data.length > 10)
so arrays of objects are properly indexed for text search.
New types: ContentType, ContentCategory, ExtractedSegment
New fields: HighlightParams.contentType, HighlightParams.contentExtractor,
Highlight.contentCategory
2026-01-27 10:27:22 -08:00
2b901974ed
chore(release): 7.7.0
2026-01-26 17:25:15 -08:00
b0ea2f3810
chore(release): 7.6.1
2026-01-26 14:22:44 -08:00
2ee6a049a6
chore(release): 7.6.0
2026-01-26 13:16:08 -08:00
f0f7acccc5
chore(release): 7.5.4
2026-01-26 13:08:44 -08:00
7883bb1545
chore(release): 7.5.3
2026-01-26 13:02:15 -08:00
1208c6597e
chore(release): 7.5.2
2026-01-26 12:52:33 -08:00
20a3a59def
chore(release): 7.5.1
2026-01-26 12:49:11 -08:00
58d85b9c91
chore(release): 7.5.0
2026-01-26 12:16:09 -08:00
a94219e720
fix: update() field asymmetry causing index corruption
...
CRITICAL: Fixed metadata index corruption on update() operations where
removalMetadata only contained custom metadata + type, while entityForIndexing
contained ALL indexed fields. This caused 7 fields to accumulate on every
update, eventually making queries return 0 results.
- Fix removalMetadata to include all indexed fields (src/brainy.ts)
- Add validateIndexConsistency() and getIndexStats() public APIs
- Add auto-corruption detection and repair on startup
- Add getOrAssignSync() for EntityIdMapper persistence
- Add comprehensive regression tests
2026-01-26 12:12:11 -08:00
478c6e6342
chore(release): 7.4.1
2026-01-20 17:38:29 -08:00
311f165533
chore(release): 7.4.0
2026-01-20 16:22:12 -08:00
24039e8a1a
chore(release): 7.3.1
2026-01-16 17:05:10 -08:00
1a813c7b86
chore(release): 7.3.0
2026-01-07 12:52:12 -08:00
19c5e67035
chore(release): 7.2.2
2026-01-07 10:46:59 -08:00
cf06d82520
chore(release): 7.2.1
2026-01-06 18:01:38 -08:00
1a32ed65ab
chore(release): 7.2.0
2026-01-06 17:19:58 -08:00
677e2d6624
perf: 580x faster embedding init - separate model from WASM
...
Cloud Run cold starts taking 139 seconds due to 90MB WASM file with
embedded 87MB model weights. WASM compilation scales with file size.
Solution: Split into 2.4MB WASM (code only) + external model files.
- WASM compile: 139,000ms → 6-8ms
- Model load: N/A → 30-115ms
- Total init: 139,000ms → 136-240ms
New modelLoader.ts handles all environments:
- Node.js: fs.readFile()
- Bun: Bun.file()
- Bun --compile: auto-embedded assets
- Browser: fetch()
Zero config - same API, npm package includes model files.
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 17:18:58 -08:00
4fc1fb7340
chore(release): 7.1.1
2026-01-06 16:03:55 -08:00
493b840527
docs: update CHANGELOG with v7.1.0 API details and performance notes
2026-01-06 15:03:02 -08:00
61100bdc9d
chore(release): 7.1.0
2026-01-06 14:53:13 -08:00
8e21e71980
chore(release): 7.0.1
2026-01-06 14:10:08 -08:00
ffd18c9598
chore(release): 7.0.0
2026-01-06 12:53:42 -08:00
81cd16e41b
chore(release): 6.6.2
2026-01-05 16:57:51 -08:00
ec6fe0c039
chore(release): 6.4.0
2025-12-11 08:41:14 -08:00
f0270cc568
chore(release): 6.3.2
2025-12-09 16:17:45 -08:00
f3765afb9e
chore(release): 6.3.1
2025-12-09 09:38:05 -08:00
e4ca3839e0
chore(release): 6.2.2
2025-11-25 15:41:12 -08:00
ebb221f8a8
perf: eliminate N+1 patterns across all APIs for 10-20x faster cloud storage
...
Fixed 8 N+1 patterns that caused severe performance degradation on cloud storage (GCS, S3, Azure, R2):
**Core Issues Fixed:**
- find(): 5 code paths loaded entities one-by-one (10x slower)
- batchGet() with vectors: Looped individual get() calls (10x slower)
- executeGraphSearch(): Loaded connected entities individually (20x slower)
- relate() duplicate check: Loaded relationships one-by-one (5x slower)
- deleteMany(): Separate transaction per entity (10x slower)
- VFS tree loading: N+1 getChildren() calls (53x slower)
- VFS file operations: updateAccessTime() write on every read (2-3x slower)
**Solutions Implemented:**
1. Batch entity loading in find() - 5 locations
- Replace individual get() with batchGet()
- GCS: 10 entities = 500ms → 50ms (10x faster)
2. Added storage.getNounBatch(ids) method
- Batch-loads vectors + metadata in parallel
- Eliminates N+1 for includeVectors: true
3. Added storage.getVerbsBatch(ids) method
- Batch-loads relationships with metadata
- Used by relate() duplicate checking
4. Added graphIndex.getVerbsBatchCached(ids)
- Cache-aware batch verb loading
- Checks UnifiedCache before storage
5. Optimized deleteMany() with transaction batching
- Chunks of 10 entities per transaction
- Atomic within chunk, graceful across chunks
6. Fixed VFS tree traversal N+1 pattern
- Graph traversal + ONE batch fetch
- 111 calls → 1 call (111x reduction)
7. Removed VFS updateAccessTime() on reads
- Eliminated 50-100ms write per read
- Follows modern filesystem noatime practice
**Performance Impact (Production GCS):**
| Operation | Before | After | Speedup |
|-----------|--------|-------|---------|
| find() 10 results | 500ms | 50ms | 10x |
| batchGet() 10 vectors | 500ms | 50ms | 10x |
| executeGraphSearch() 20 | 1000ms | 50ms | 20x |
| relate() duplicate (5) | 250ms | 50ms | 5x |
| deleteMany() 10 entities | 2000ms | 200ms | 10x |
| VFS tree loading | 5304ms | 100ms | 53x |
| VFS readFile() | 100-150ms | 50ms | 2-3x |
**Architecture:**
- All batch methods use readBatchWithInheritance() for COW/fork/asOf support
- Works with all storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem)
- Cache-aware with proper UnifiedCache integration
- Transaction-safe with atomic chunked operations
- Fully backward compatible
**Files Modified:**
- src/brainy.ts: Fixed find(), batchGet(), relate(), deleteMany(), executeGraphSearch()
- src/storage/baseStorage.ts: Added getNounBatch(), getVerbsBatch()
- src/graph/graphAdjacencyIndex.ts: Added getVerbsBatchCached()
- src/vfs/VirtualFileSystem.ts: Fixed tree traversal, removed updateAccessTime()
- src/coreTypes.ts: Added batch method signatures to StorageAdapter
- src/types/brainy.types.ts: Added continueOnError to DeleteManyParams
- tests/: Added comprehensive regression tests
**Overall Impact:**
- 10-20x faster batch operations on cloud storage
- 50-90% cost reduction (fewer storage API calls)
- Production-ready with clean architecture
- Zero breaking changes - automatic performance improvement
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 15:18:26 -08:00
