Add confidence and weight properties to Entity interface and flatten Result fields to top level for improved developer experience and API consistency.
Breaking Changes: None (all changes are backward compatible)
Phase 2 - Entity Confidence & Weight:
- Add confidence (type classification certainty) and weight (entity importance) to Entity interface
- Add confidence/weight parameters to AddParams and UpdateParams
- Update convertNounToEntity() to extract confidence/weight from storage
- Update add() and update() methods to preserve confidence/weight in metadata
- Enable developers to specify and access entity confidence/weight scores
Phase 3 - Result Field Flattening:
- Flatten commonly-used entity fields (type, metadata, data, confidence, weight) to Result top level
- Add createResult() helper for consistent Result construction
- Update all find() code paths to use createResult()
- Enable direct access: result.metadata instead of result.entity.metadata
- Preserve full entity in result.entity for backward compatibility
VFS Fix (from previous work):
- Fix VFSStructureGenerator to use brain.vfs() cached instance instead of creating separate instance
- Improve VFS error messages with step-by-step guidance
- Update examples to show correct vfs.init() usage
- Add comprehensive VFS import verification tests
Documentation Updates:
- Update API_REFERENCE.md with confidence/weight examples and flattened Result documentation
- Enhance JSDoc for add(), get(), find(), similar() with v4.3.0 examples
- Document Result structure changes and backward compatibility
- Add migration examples showing both old and new access patterns
Tests:
- Add 16 comprehensive tests for Entity confidence/weight exposure
- Add tests for Result field flattening
- Add tests for backward compatibility
- All tests passing (16/16)
API Consistency:
- Entity: direct access to confidence/weight
- Result: flattened fields + nested entity (both work)
- Relation: already had confidence/weight (consistent)
- VFS: inherits from Entity (automatic)
Files Changed:
- src/types/brainy.types.ts - Updated Entity, AddParams, UpdateParams, Result interfaces
- src/brainy.ts - Updated implementation and JSDoc for all affected methods
- tests/integration/entity-confidence-weight.test.ts - 16 comprehensive tests
- docs/API_REFERENCE.md - Updated with v4.3.0 examples
- src/importers/VFSStructureGenerator.ts - VFS fix
- src/vfs/VirtualFileSystem.ts - Improved error messages
- examples/unified-import-example.ts - Added vfs.init() example
- tests/integration/vfs-*-verification.test.ts - VFS verification tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Removed ImportManager class and exports (use brain.import() instead)
- Fixed all documentation: getStatistics() → getStats()
- Updated 41 files across codebase for consistency
- Removed ImportManager section from API docs
- Added v3.30.0 migration guide to CHANGELOG
Co-Authored-By: Claude <noreply@anthropic.com>
Improve documentation to explicitly show that:
- type: 'gcs' requires gcsStorage config (S3-compatible, HMAC)
- type: 'gcs-native' requires gcsNativeStorage config (native SDK, ADC)
- Mixing type and config objects will fall back to memory storage
- Auto-detection works when type is omitted
Added 'Common Mistakes' section with examples of incorrect usage.
Enhanced migration guide with step-by-step instructions.
Fixes critical scalability bottleneck where metadata was stored in non-sharded
directories, causing performance degradation at scale (1M+ entities).
Changes:
- Add UUID-based sharding to metadata operations in S3Compatible, FileSystem, and OpFS storage
- Implement complete UUID-based sharding for OpFS storage (nouns, verbs, metadata)
- Update pagination methods to iterate through all 256 UUID-based shards
- Add integration tests verifying sharding behavior across storage adapters
Impact:
- Metadata now scales to millions of entities without directory bottlenecks
- All storage adapters now use consistent UUID-based sharding (256 buckets: 00-ff)
- Improves GCS/S3/R2/OpFS performance at scale
- Path format: entities/{type}/{subtype}/{shard}/{id}.json
Breaking change: Requires data migration for existing S3/GCS/R2/OpFS deployments.
See .strategy/UNIFIED-UUID-SHARDING.md for migration guidance.
Resolves API accessibility issues reported by Brain Cloud Studio team:
1. Export ImportManager and createImportManager
- ImportManager class was fully implemented but not exported
- Consumers can now access AI-powered import features
- Includes ImportOptions and ImportResult types
2. Add brain.getStats() convenience method
- Documentation showed getStats() as top-level method
- Implementation had it nested under brain.counts.getStats()
- Added convenience method that delegates to counts API
- Maintains backward compatibility
3. Update API documentation
- Corrected getStats() signature and return type
- Added comprehensive ImportManager documentation
- Included examples for all import patterns
These changes expose existing, tested functionality without
modifying core behavior. All tests pass.
Add IntelligentImportAugmentation with support for:
- CSV: auto-detection of encoding, delimiters, and field types
- Excel: multi-sheet extraction with metadata preservation
- PDF: text extraction, table detection, and metadata extraction
Features:
- Automatic format detection from file extension or content
- Intelligent type inference (string, number, boolean, date)
- Seamless integration with neural entity extraction
- Production-ready with 69 comprehensive tests
Dependencies added:
- xlsx@^0.18.5 for Excel parsing
- pdfjs-dist@^4.0.379 for PDF parsing
- csv-parse@^6.1.0 for CSV parsing
- chardet@^2.0.0 for encoding detection
Documentation:
- Updated README with import examples
- Updated API_REFERENCE with comprehensive import() docs
- Updated import-anything.md guide
- Added working example: import-excel-pdf-csv.ts
- Updated augmentations README
- Add @deprecated JSDoc tags to TypeScript definitions
- Update all documentation examples to use modern add() and relate() API
- Preserve batch operations (addNouns, addVerbs) as they remain current
- Mark deprecated methods in both source and compiled definitions
Migration guide:
- addNoun(data, type, metadata) → add(data, { nounType: type, ...metadata })
- addVerb(source, target, type, metadata) → relate(source, target, type, metadata)
- Implement self-configuring validation that adapts to system resources
- Add validation for all CRUD operations (add, update, delete, find, relate)
- Auto-configure limits based on available memory (1GB = 10K limit, 8GB = 80K)
- Monitor and auto-tune performance based on query response times
- Fix multiple type filtering with proper anyOf structure
- Enhance type safety by requiring NounType/VerbType enums
- Fix tests to validate correct behavior (no fake implementations)
- Add comprehensive VALIDATION.md documentation
- Update API_REFERENCE.md with validation rules and examples
- Clarify metadata update behavior (null keeps existing, {} clears)
BREAKING CHANGE: getFieldsForType() now requires NounType enum instead of string
Co-Authored-By: Claude <noreply@anthropic.com>
