soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

**test(api): add integration tests for API endpoints and core functionality**

Browse source 
- **Integration Tests**:
  - Introduced `api-integration.test.ts` to validate API functionality:
    - Verifies text insertion, vector embedding generation, and search operations.
    - Confirms HNSW index correctness for vector similarity search.
    - Ensures no dimensional mismatches in embeddings.

- **Test Server**:
  - Added test server utilizing Express for endpoint simulation (`/insert` and `/search/text`).

- **Dependencies**:
  - Introduced `express` and `node-fetch` as new dependencies for testing purposes.

- **Vitest Fix**:
  - Updated `vitest.config.ts` to resolve the `process.memoryUsage` error by setting `logHeapUsage: false`.

- **Package Updates**:
  - Modified `package-lock.json` to include newly added dependencies and updates.

**Purpose**: Guarantees the stability of core API endpoints and vector-related functionality, ensuring reliable behavior for end-to-end scenarios.
This commit is contained in:
David Snelling 2025-07-28 10:04:45 -07:00
parent 5d607068a8
commit 7f082df6a3
11 changed files with 1418 additions and 85 deletions
Download patch file Download diff file Expand all files Collapse all files

68
CHANGES.md
View file

@ -1,40 +1,50 @@
# Changes Made to Fix S3 Storage Tests
# Changes
## Issues Identified
## 2025-07-28
The S3 storage tests were failing due to several issues:
### Bug Fixes
1. **Metadata Operations**: The metadata was not being correctly retrieved from the mock S3 storage.
2. **Noun Operations**: The nouns were not being correctly retrieved from the mock S3 storage, with the ID property being undefined.
3. **Verb Operations**: The verbs were not being correctly retrieved from the mock S3 storage, with the ID property being undefined.
4. **Storage Status**: The storage usage was being reported as 0 even when objects were stored.
5. **Multiple Objects**: The test was expecting 10 nouns to be retrieved, but it was retrieving 0.
- Fixed an issue in FileSystemStorage constructor where path operations were performed before the path module was fully loaded. The fix defers path operations until the init() method is called, when the path module is guaranteed to be loaded.
## Changes Made
### Details
### S3 Storage Adapter (`src/storage/adapters/s3CompatibleStorage.ts`)
The issue was in the FileSystemStorage constructor where it was using the path module synchronously:
1. **Enhanced Logging**: Added more detailed logging to help diagnose issues.
2. **Improved Error Handling**: Added better error handling and logging for edge cases.
3. **Storage Status Calculation**: Ensured that the storage status calculation always returns a positive size if there are any objects in the storage.
```typescript
constructor(rootDirectory: string) {
super()
this.rootDir = rootDirectory
this.nounsDir = path.join(this.rootDir, NOUNS_DIR) // Error here - path could be undefined
this.verbsDir = path.join(this.rootDir, VERBS_DIR)
this.metadataDir = path.join(this.rootDir, METADATA_DIR)
this.indexDir = path.join(this.rootDir, INDEX_DIR)
}
```
### S3 Mock Implementation (`tests/mocks/s3-mock.ts`)
However, the path module was being loaded asynchronously via dynamic imports:
1. **Object Structure**: Added the `contentType` property to the `S3MockObject` interface to ensure that the mock objects have the same structure as real S3 objects.
2. **Object Persistence**: Ensured that objects are correctly persisted between operations by using a global `mockS3Storage` variable.
3. **Enhanced Logging**: Added more detailed logging to help diagnose issues.
4. **Object Validation**: Added validation to ensure that objects have the required properties, particularly the `id` property for nouns and verbs.
5. **Reset Function**: Enhanced the `reset` function to provide more detailed logging about the state of the mock storage before and after reset.
6. **Mock Client Creation**: Modified the `createMockS3Client` function to ensure that it's using the same instance of `mockS3Storage` for all operations.
```typescript
try {
// Using dynamic imports to avoid issues in browser environments
const fsPromise = import('fs')
const pathPromise = import('path')
## Why These Changes Fixed the Issues
Promise.all([fsPromise, pathPromise]).then(([fsModule, pathModule]) => {
fs = fsModule
path = pathModule.default
}).catch(error => {
console.error('Failed to load Node.js modules:', error)
})
} catch (error) {
console.error(
'FileSystemStorage: Failed to load Node.js modules. This adapter is not supported in this environment.',
error
)
}
```
1. **Metadata Operations**: The enhanced logging helped identify that the metadata was being correctly stored but not correctly retrieved. Adding the `contentType` property to the response object fixed this issue.
2. **Noun Operations**: The validation added to ensure that objects have the required properties, particularly the `id` property, fixed the issue with nouns not being correctly retrieved.
3. **Verb Operations**: Similar to noun operations, the validation added to ensure that objects have the required properties fixed the issue with verbs not being correctly retrieved.
4. **Storage Status**: The change to ensure that the storage status calculation always returns a positive size if there are any objects in the storage fixed this issue.
5. **Multiple Objects**: The changes to ensure that objects are correctly persisted between operations and that the mock client is using the same instance of `mockS3Storage` for all operations fixed this issue.
The fix:
1. Modified the constructor to only store the rootDirectory and defer path operations
2. Updated the init() method to initialize directory paths when the path module is guaranteed to be loaded
## Conclusion
The S3 storage tests are now passing. The changes made to the S3 storage adapter and the S3 mock implementation have successfully fixed the issues with the tests.
This ensures that path operations are only performed when the path module is available, preventing the "Cannot read properties of undefined (reading 'join')" error.