2025-07-21 12:46:41 -07:00
/ * *
* Base Storage Adapter
* Provides common functionality for all storage adapters
* /
**feat(core, storage, tests): add service-level statistics tracking and storage adapter enhancements**
- **Core**: Enhanced `getStatistics` function to support `service` and `service[]` filters, enabling statistics breakdown by service. Modified return structure to include `serviceBreakdown` for detailed insights.
- **Storage**: Implemented a new `BaseStorageAdapter` abstract class to centralize statistics-related functionality, such as incrementing/decrementing counters and updating HNSW index size. Refactored all storage adapters (`FileSystemStorage`, `S3CompatibleStorage`, `MemoryStorage`, `OPFSStorage`) to extend `BaseStorageAdapter`, ensuring consistent statistics tracking.
- **Tests**: Added new test cases in `statistics.test.ts` to validate service-level statistics tracking, breakdown accuracy, and multi-service filtering.
**Purpose**: Improve insight into data trends by tracking service-specific usage in statistics. Enhance maintainability and consistency through storage adapter centralization and robust testing.
2025-07-24 11:35:52 -07:00
import { GraphVerb , HNSWNoun , StatisticsData } from '../coreTypes.js'
import { BaseStorageAdapter } from './adapters/baseStorageAdapter.js'
2025-07-21 12:46:41 -07:00
// Common directory/prefix names
export const NOUNS_DIR = 'nouns'
export const VERBS_DIR = 'verbs'
export const METADATA_DIR = 'metadata'
export const INDEX_DIR = 'index'
**feat(core, storage, tests): add service-level statistics tracking and storage adapter enhancements**
- **Core**: Enhanced `getStatistics` function to support `service` and `service[]` filters, enabling statistics breakdown by service. Modified return structure to include `serviceBreakdown` for detailed insights.
- **Storage**: Implemented a new `BaseStorageAdapter` abstract class to centralize statistics-related functionality, such as incrementing/decrementing counters and updating HNSW index size. Refactored all storage adapters (`FileSystemStorage`, `S3CompatibleStorage`, `MemoryStorage`, `OPFSStorage`) to extend `BaseStorageAdapter`, ensuring consistent statistics tracking.
- **Tests**: Added new test cases in `statistics.test.ts` to validate service-level statistics tracking, breakdown accuracy, and multi-service filtering.
**Purpose**: Improve insight into data trends by tracking service-specific usage in statistics. Enhance maintainability and consistency through storage adapter centralization and robust testing.
2025-07-24 11:35:52 -07:00
export const STATISTICS_KEY = 'statistics'
2025-07-21 12:46:41 -07:00
/ * *
* Base storage adapter that implements common functionality
* This is an abstract class that should be extended by specific storage adapters
* /
**feat(core, storage, tests): add service-level statistics tracking and storage adapter enhancements**
- **Core**: Enhanced `getStatistics` function to support `service` and `service[]` filters, enabling statistics breakdown by service. Modified return structure to include `serviceBreakdown` for detailed insights.
- **Storage**: Implemented a new `BaseStorageAdapter` abstract class to centralize statistics-related functionality, such as incrementing/decrementing counters and updating HNSW index size. Refactored all storage adapters (`FileSystemStorage`, `S3CompatibleStorage`, `MemoryStorage`, `OPFSStorage`) to extend `BaseStorageAdapter`, ensuring consistent statistics tracking.
- **Tests**: Added new test cases in `statistics.test.ts` to validate service-level statistics tracking, breakdown accuracy, and multi-service filtering.
**Purpose**: Improve insight into data trends by tracking service-specific usage in statistics. Enhance maintainability and consistency through storage adapter centralization and robust testing.
2025-07-24 11:35:52 -07:00
export abstract class BaseStorage extends BaseStorageAdapter {
2025-07-21 12:46:41 -07:00
protected isInitialized = false
/ * *
* Initialize the storage adapter
* This method should be implemented by each specific adapter
* /
public abstract init ( ) : Promise < void >
/ * *
* Ensure the storage adapter is initialized
* /
protected async ensureInitialized ( ) : Promise < void > {
if ( ! this . isInitialized ) {
await this . init ( )
}
}
/ * *
* Save a noun to storage
* /
public async saveNoun ( noun : HNSWNoun ) : Promise < void > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . saveNoun_internal ( noun )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get a noun from storage
* /
public async getNoun ( id : string ) : Promise < HNSWNoun | null > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getNoun_internal ( id )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get all nouns from storage
* /
public async getAllNouns ( ) : Promise < HNSWNoun [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getAllNouns_internal ( )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get nouns by noun type
* @param nounType The noun type to filter by
* @returns Promise that resolves to an array of nouns of the specified noun type
* /
public async getNounsByNounType ( nounType : string ) : Promise < HNSWNoun [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getNounsByNounType_internal ( nounType )
2025-07-21 12:46:41 -07:00
}
/ * *
* Delete a noun from storage
* /
public async deleteNoun ( id : string ) : Promise < void > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . deleteNoun_internal ( id )
2025-07-21 12:46:41 -07:00
}
/ * *
* Save a verb to storage
* /
public async saveVerb ( verb : GraphVerb ) : Promise < void > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . saveVerb_internal ( verb )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get a verb from storage
* /
public async getVerb ( id : string ) : Promise < GraphVerb | null > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getVerb_internal ( id )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get all verbs from storage
* /
public async getAllVerbs ( ) : Promise < GraphVerb [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getAllVerbs_internal ( )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get verbs by source
* /
public async getVerbsBySource ( sourceId : string ) : Promise < GraphVerb [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getVerbsBySource_internal ( sourceId )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get verbs by target
* /
public async getVerbsByTarget ( targetId : string ) : Promise < GraphVerb [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getVerbsByTarget_internal ( targetId )
2025-07-21 12:46:41 -07:00
}
/ * *
* Get verbs by type
* /
public async getVerbsByType ( type : string ) : Promise < GraphVerb [ ] > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . getVerbsByType_internal ( type )
2025-07-21 12:46:41 -07:00
}
/ * *
* Delete a verb from storage
* /
public async deleteVerb ( id : string ) : Promise < void > {
await this . ensureInitialized ( )
2025-07-25 09:44:10 -07:00
return this . deleteVerb_internal ( id )
2025-07-21 12:46:41 -07:00
}
/ * *
* Clear all data from storage
* This method should be implemented by each specific adapter
* /
public abstract clear ( ) : Promise < void >
/ * *
* Get information about storage usage and capacity
* This method should be implemented by each specific adapter
* /
public abstract getStorageStatus ( ) : Promise < {
type : string
used : number
quota : number | null
details? : Record < string , any >
} >
/ * *
* Save metadata to storage
* This method should be implemented by each specific adapter
* /
public abstract saveMetadata ( id : string , metadata : any ) : Promise < void >
/ * *
* Get metadata from storage
* This method should be implemented by each specific adapter
* /
public abstract getMetadata ( id : string ) : Promise < any | null >
/ * *
2025-07-25 09:44:10 -07:00
* Save a noun to storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract saveNoun_internal ( noun : HNSWNoun ) : Promise < void >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get a noun from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getNoun_internal ( id : string ) : Promise < HNSWNoun | null >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get all nouns from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getAllNouns_internal ( ) : Promise < HNSWNoun [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get nouns by noun type
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getNounsByNounType_internal ( nounType : string ) : Promise < HNSWNoun [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Delete a noun from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract deleteNoun_internal ( id : string ) : Promise < void >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Save a verb to storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract saveVerb_internal ( verb : GraphVerb ) : Promise < void >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get a verb from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getVerb_internal ( id : string ) : Promise < GraphVerb | null >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get all verbs from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getAllVerbs_internal ( ) : Promise < GraphVerb [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get verbs by source
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getVerbsBySource_internal ( sourceId : string ) : Promise < GraphVerb [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get verbs by target
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getVerbsByTarget_internal ( targetId : string ) : Promise < GraphVerb [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Get verbs by type
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract getVerbsByType_internal ( type : string ) : Promise < GraphVerb [ ] >
2025-07-21 12:46:41 -07:00
/ * *
2025-07-25 09:44:10 -07:00
* Delete a verb from storage
2025-07-21 12:46:41 -07:00
* This method should be implemented by each specific adapter
* /
2025-07-25 09:44:10 -07:00
protected abstract deleteVerb_internal ( id : string ) : Promise < void >
2025-07-21 12:46:41 -07:00
/ * *
* Helper method to convert a Map to a plain object for serialization
* /
protected mapToObject < K extends string | number , V > (
map : Map < K , V > ,
valueTransformer : ( value : V ) = > any = ( v ) = > v
) : Record < string , any > {
const obj : Record < string , any > = { }
for ( const [ key , value ] of map . entries ( ) ) {
obj [ key . toString ( ) ] = valueTransformer ( value )
}
return obj
}
**feat(core, storage, tests): add service-level statistics tracking and storage adapter enhancements**
- **Core**: Enhanced `getStatistics` function to support `service` and `service[]` filters, enabling statistics breakdown by service. Modified return structure to include `serviceBreakdown` for detailed insights.
- **Storage**: Implemented a new `BaseStorageAdapter` abstract class to centralize statistics-related functionality, such as incrementing/decrementing counters and updating HNSW index size. Refactored all storage adapters (`FileSystemStorage`, `S3CompatibleStorage`, `MemoryStorage`, `OPFSStorage`) to extend `BaseStorageAdapter`, ensuring consistent statistics tracking.
- **Tests**: Added new test cases in `statistics.test.ts` to validate service-level statistics tracking, breakdown accuracy, and multi-service filtering.
**Purpose**: Improve insight into data trends by tracking service-specific usage in statistics. Enhance maintainability and consistency through storage adapter centralization and robust testing.
2025-07-24 11:35:52 -07:00
/ * *
* Save statistics data to storage
* This method should be implemented by each specific adapter
* @param statistics The statistics data to save
* /
protected abstract saveStatisticsData ( statistics : StatisticsData ) : Promise < void >
/ * *
* Get statistics data from storage
* This method should be implemented by each specific adapter
* @returns Promise that resolves to the statistics data or null if not found
* /
protected abstract getStatisticsData ( ) : Promise < StatisticsData | null >
2025-07-21 12:46:41 -07:00
}