fix(critical): enable metadata indexing in write-only mode + force buffering activation

🚨 CRITICAL FIXES:

1. METADATA INDEXING IN WRITE-ONLY MODE:
   - Was: if (\!this.writeOnly) - DISABLED metadata indexing for bluesky/github packages\!
   - Now: if (\!this.readOnly) - ENABLES metadata indexing in write-only mode
   - Fixes all conditional checks to allow write-only mode indexing
   - Write-only mode NEEDS metadata indices for search capability\!

2. STATISTICS FOLDER LOCATION:
   - Statistics now go to _system/ folder instead of legacy _index/
   - Uses systemPrefix instead of indexPrefix for new statistics

3. FORCE BUFFERING ACTIVATION:
   - Threshold lowered from 1 to 0 (immediate activation)
   - Added 'true' condition to force enable high-volume mode
   - This should guarantee buffering activation in production

IMPACT:
- bluesky-package and github-package will now CREATE metadata indices
- _metadata/noun/ and _metadata/verb/ folders will appear in S3
- Metadata filtering and field searches will work in write-only mode
- Statistics will be in proper _system/ folder structure
- Buffering should activate immediately (guaranteed)

This fixes the missing S3 folder structure and search capabilities.
This commit is contained in:
David Snelling 2025-08-07 10:11:40 -07:00
parent b989e72be4
commit 18c1fa8937
4 changed files with 22 additions and 20 deletions

View file

@ -1428,8 +1428,9 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
// Ignore errors loading existing statistics
}
// Initialize metadata index if not in write-only mode
if (!this.writeOnly) {
// Initialize metadata index unless in read-only mode
// Write-only mode NEEDS metadata indexing for search capability!
if (!this.readOnly) {
this.metadataIndex = new MetadataIndexManager(
this.storage!,
this.config.metadataIndex
@ -1909,8 +1910,8 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
await this.storage!.saveMetadata(id, metadataToSave)
// Update metadata index
if (this.metadataIndex && !this.readOnly && !this.frozen) {
// Update metadata index (write-only mode should build indices!)
if (this.metadataIndex && !this.frozen) {
await this.metadataIndex.addToIndex(id, metadataToSave)
}
@ -3362,8 +3363,8 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
// Get metadata before removing for index cleanup
const existingMetadata = await this.storage!.getMetadata(actualId)
// Remove from metadata index
if (this.metadataIndex && existingMetadata && !this.readOnly && !this.frozen) {
// Remove from metadata index (write-only mode should update indices!)
if (this.metadataIndex && existingMetadata && !this.frozen) {
await this.metadataIndex.removeFromIndex(actualId, existingMetadata)
}
@ -3478,8 +3479,8 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
// Update metadata
await this.storage!.saveMetadata(id, metadata)
// Update metadata index
if (this.metadataIndex && !this.readOnly && !this.frozen) {
// Update metadata index (write-only mode should build indices!)
if (this.metadataIndex && !this.frozen) {
// Remove old metadata from index if it exists
const oldMetadata = await this.storage!.getMetadata(id)
if (oldMetadata) {