soulcraftlabs/open-brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 342 Packages Wiki Activity Actions 6
open-brainy/src/utils
History
David Snelling 6161ce3f5e perf(metadata-index): implement adaptive batch sizing for first-run rebuilds 
- Issue: v4.2.1 field registry only helps on 2nd+ runs - first run still slow (8-9 min for 1,157 entities)
- Root Cause: Batch size of 25 was designed for cloud storage socket exhaustion, too conservative for local storage
- Solution: Adaptive batch sizing based on storage adapter type
  - FileSystemStorage/MemoryStorage/OPFSStorage: 500 items/batch (fast local I/O, no socket limits)
  - GCS/S3/R2 (cloud storage): 25 items/batch (prevent socket exhaustion)
- Performance Impact:
  - FileSystem first-run rebuild: 8-9 min → 30-60 seconds (10-15x faster)
  - 1,157 entities: 46 batches @ 25 → 3 batches @ 500 (15x fewer I/O operations)
  - Cloud storage: No change (still 25/batch for safety)
- Detection: Auto-detects storage type via constructor.name
- Zero Config: Completely automatic, no configuration needed
- Combined with v4.2.1: First run fast, subsequent runs instant (2-3 sec)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 09:02:37 -07:00
..
adaptiveBackpressure.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
adaptiveSocketManager.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
autoConfiguration.ts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
BoundedRegistry.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
brainyTypes.ts feat: implement comprehensive type safety system with BrainyTypes API 2025-09-01 09:37:36 -07:00
crypto.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
deletedItemsIndex.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
distance.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
embedding.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
enhancedLogger.ts feat: Brainy 3.0 - Triple Intelligence Release 2025-09-15 11:06:16 -07:00
ensureDeleted.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
entityIdMapper.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
environment.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
fieldNameTracking.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
fieldTypeInference.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
index.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
intelligentTypeMapper.ts feat: add distributed scaling and enterprise features for v3 2025-09-08 14:26:09 -07:00
jsonProcessing.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
logger.ts feat: add complete silent mode for TUI applications 2025-09-16 13:18:49 -07:00
memoryDetection.ts feat: implement always-adaptive caching with getCacheStats monitoring 2025-10-10 14:09:30 -07:00
metadataFilter.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataIndex.ts perf(metadata-index): implement adaptive batch sizing for first-run rebuilds 2025-10-23 09:02:37 -07:00
metadataIndexCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
metadataIndexChunking.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
metadataNamespace.ts feat: comprehensive metadata namespace architecture and cleanup system 2025-08-27 15:38:48 -07:00
mutex.ts feat: complete VFS with Knowledge Layer integration 2025-09-25 10:47:44 -07:00
nodeVersionCheck.ts fix: resolve browser compatibility by avoiding Node.js process access in nodeVersionCheck 2025-09-17 16:59:59 -07:00
operationUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
paramValidation.ts fix: resolve 10 test failures across clustering, metadata, and deletion 2025-10-09 16:33:08 -07:00
performanceMonitor.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
periodicCleanup.ts feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
rateLimiter.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
rebuildCounts.ts fix(storage): resolve count synchronization race condition across all storage adapters 2025-10-21 11:19:08 -07:00
requestCoalescer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
requestDeduplicator.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
searchCache.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
statisticsCollector.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
structuredLogger.ts feat: add browser environment compatibility support 2025-09-17 15:48:02 -07:00
textEncoding.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
typeUtils.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
typeValidation.ts feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
unifiedCache.ts perf: more aggressive cache fairness to prevent thrashing 2025-10-13 11:50:53 -07:00
version.ts fix: update hardcoded version references to use dynamic version 2025-09-26 13:41:16 -07:00
workerUtils.ts fix: correct Node.js version references from 24 to 22 in comments and code 2025-10-20 11:43:31 -07:00
writeBuffer.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00