soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions
brainy/src/utils
History
David Snelling daf33b9e9b fix(metadata-index): fix rebuild stalling after first batch on FileSystemStorage 
- Critical Fix: v4.2.2 rebuild stalled after processing first batch (500/1,157 entities)
- Root Cause: getAllShardedFiles() was called on EVERY batch, re-reading all 256 shard directories each time
- Performance Impact: Second batch call to getAllShardedFiles() took 3+ minutes, appearing to hang
- Solution: Load all entities at once for local storage (FileSystem/Memory/OPFS)
  - FileSystem/Memory/OPFS: Load all nouns/verbs in single batch (no pagination overhead)
  - Cloud (GCS/S3/R2): Keep conservative pagination (25 items/batch for socket safety)
- Benefits:
  - FileSystem: 1,157 entities load in 2-3 seconds (one getAllShardedFiles() call)
  - Cloud: Unchanged behavior (still uses safe batching)
  - Zero config: Auto-detects storage type via constructor.name
- Technical Details:
  - Pagination was designed for cloud storage socket exhaustion
  - FileSystem doesn't need pagination - can handle loading thousands of entities at once
  - Eliminates repeated directory scans: 3 batches × 256 dirs → 1 batch × 256 dirs
- Workshop Team: This resolves the v4.2.2 stalling issue - rebuild will now complete in seconds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 09:19:17 -07:00
..
adaptiveBackpressure.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
adaptiveSocketManager.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
autoConfiguration.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
BoundedRegistry.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
brainyTypes.ts feat: implement comprehensive type safety system with BrainyTypes API 2025-09-01 09:37:36 -07:00
crypto.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
deletedItemsIndex.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
distance.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
embedding.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
enhancedLogger.ts feat: Brainy 3.0 - Triple Intelligence Release 2025-09-15 11:06:16 -07:00
ensureDeleted.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
entityIdMapper.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
environment.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
fieldNameTracking.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
fieldTypeInference.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
index.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
intelligentTypeMapper.ts feat: add distributed scaling and enterprise features for v3 2025-09-08 14:26:09 -07:00
jsonProcessing.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
logger.ts feat: add complete silent mode for TUI applications 2025-09-16 13:18:49 -07:00
memoryDetection.ts feat: implement always-adaptive caching with getCacheStats monitoring 2025-10-10 14:09:30 -07:00
metadataFilter.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataIndex.ts fix(metadata-index): fix rebuild stalling after first batch on FileSystemStorage 2025-10-23 09:19:17 -07:00
metadataIndexCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
metadataIndexChunking.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataNamespace.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
mutex.ts feat: complete VFS with Knowledge Layer integration 2025-09-25 10:47:44 -07:00
nodeVersionCheck.ts fix: resolve browser compatibility by avoiding Node.js process access in nodeVersionCheck 2025-09-17 16:59:59 -07:00
operationUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
paramValidation.ts fix: resolve 10 test failures across clustering, metadata, and deletion 2025-10-09 16:33:08 -07:00
performanceMonitor.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
periodicCleanup.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
rateLimiter.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
rebuildCounts.ts fix(storage): resolve count synchronization race condition across all storage adapters 2025-10-21 11:19:08 -07:00
requestCoalescer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
requestDeduplicator.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
searchCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
statisticsCollector.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
structuredLogger.ts feat: add browser environment compatibility support 2025-09-17 15:48:02 -07:00
textEncoding.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
typeUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
typeValidation.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
unifiedCache.ts perf: more aggressive cache fairness to prevent thrashing 2025-10-13 11:50:53 -07:00
version.ts fix: update hardcoded version references to use dynamic version 2025-09-26 13:41:16 -07:00
workerUtils.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
writeBuffer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00