- **Core**: Improved verb creation logic by adding `createdAt`, `updatedAt`, and `createdBy` attributes. These fields include timestamped metadata (`seconds`, `nanoseconds`) and source augmentation/service information for better tracking.
- **Storage**: Refactored `BaseStorage` methods to utilize internal variants (e.g., `saveVerb_internal`, `getNoun_internal`). Added support for new verb attributes while maintaining backward compatibility with existing data structures.
- **Tests**:
- Updated `s3-storage.test.ts` and `opfs-storage.test.ts` to validate changes in verb attributes such as timestamps and augmentation metadata.
- Added assertions for `createdAt`, `updatedAt`, and `createdBy` fields in test cases.
- **Cleanup**: Replaced ambiguous type aliases like `Edge` and `HNSWNode` with clearer equivalents (`Verb` and `HNSWNoun_internal`) for consistency across storage adapters.
**Purpose**: Enhance metadata tracking and standardize attribute handling across storage and core modules to ensure accurate and consistent data throughout the system.
- **Tests**: Added new `statistics-storage.test.ts` to validate statistics storage functionality across scenarios including saving, retrieving, time-based partitioning, and backward compatibility. Ensured tests dynamically handle missing environment variables by skipping S3-related tests when credentials are unavailable.
- **Docs**: Enhanced `statistics.md` with detailed explanations of scalability improvements, including adaptive flush timing, batched updates, and time-based partitioning. Improved readability and structure.
- **Storage**: Updated all storage adapters to integrate time-based partitioning and maintain backward compatibility with legacy statistics storage formats.
- **Dependencies**: Added `dotenv` to support environmental variable management for storage adapter tests.
**Purpose**: Strengthen system reliability by adding comprehensive test coverage for statistics storage, improve scalability documentation, and ensure consistency across storage adapters with robust implementations.
- **Core**: Enhanced `getStatistics` function to support `service` and `service[]` filters, enabling statistics breakdown by service. Modified return structure to include `serviceBreakdown` for detailed insights.
- **Storage**: Implemented a new `BaseStorageAdapter` abstract class to centralize statistics-related functionality, such as incrementing/decrementing counters and updating HNSW index size. Refactored all storage adapters (`FileSystemStorage`, `S3CompatibleStorage`, `MemoryStorage`, `OPFSStorage`) to extend `BaseStorageAdapter`, ensuring consistent statistics tracking.
- **Tests**: Added new test cases in `statistics.test.ts` to validate service-level statistics tracking, breakdown accuracy, and multi-service filtering.
**Purpose**: Improve insight into data trends by tracking service-specific usage in statistics. Enhance maintainability and consistency through storage adapter centralization and robust testing.
- Introduced comprehensive mock for the Origin Private File System (OPFS) in `tests/mocks/opfs-mock.ts`, simulating environment for detailed storage testing.
- Added `tests/opfs-storage.test.ts`, containing extensive test cases for `OPFSStorage` operations including metadata, nouns, verbs, and storage status.
- Improved S3 mock implementation in `tests/mocks/s3-mock.ts` with better object persistence, validation, and logging to emulate real S3 behavior.
- Resolved issues related to metadata, nouns, verbs, and storage usage inconsistencies in mock storage adapters.
- Enhanced logging and error handling to aid in debugging and test reliability.
Purpose: Improve test completeness and reliability by introducing detailed mocks and extended test cases for S3 and OPFS storage systems.
- Added `BaseStorage` abstract class to provide common functionality for storage adapters.
- Implemented `FileSystemStorage` adapter for Node.js environments, supporting persistent file-based storage.
- Added directory structure management methods to `FileSystemStorage` (`ensureDirectoryExists`, `clear`, etc.).
- Implemented `MemoryStorage` adapter for in-memory storage suitable for non-persistent environments.
- Supported CRUD operations for nouns, verbs, and metadata in both `FileSystemStorage` and `MemoryStorage`.
- Included utility methods like `mapToObject` for serialization and deserialization of complex data structures.
Purpose: Provide extensible, reusable storage solutions with both persistent and non-persistent options for diverse application needs.
