brainy/docs/user-guides/WRITEONLY_MODE_IMPLEMENTATION.md

Write-Only Mode Implementation Summary

Overview

This implementation addresses the GitHub issue regarding write-only mode behavior in Brainy, specifically:

1. Enabling existence checks in write-only mode via direct storage queries
2. Improving placeholder noun handling to avoid indexing mock data
3. Ensuring auto-configuration works seamlessly

Changes Made

1. Enhanced get() Method for Write-Only Mode

File: src/brainyData.ts (lines 2084-2105)

• Modified to query storage directly when in write-only mode since index is not loaded
• Added fallback logic for normal mode to check storage if item not found in index
• Maintains backward compatibility while enabling existence checks in write-only mode

// In write-only mode, query storage directly since index is not loaded
if (this.writeOnly) {
  try {
    noun = await this.storage!.getNoun(id)
  } catch (storageError) {
    return null
  }
} else {
  // Normal mode: Get noun from index first, fallback to storage
  noun = this.index.getNouns().get(id)
  if (!noun && this.storage) {
    try {
      noun = await this.storage.getNoun(id)
    } catch (storageError) {
      return null
    }
  }
}

2. Enhanced add() Method for Existence Checks

File: src/brainyData.ts (lines 1211-1247)

• Added comprehensive existence checking for both write-only and normal modes
• Detects and handles placeholder noun replacement when real data is provided
• Skips index operations in write-only mode while maintaining storage operations

// Check for existing noun (both write-only and normal modes)
let existingNoun: HNSWNoun | undefined
if (options.id) {
  // Check if existing noun is a placeholder and replace with real data
  const isPlaceholder = existingMetadata && 
    typeof existingMetadata === 'object' && 
    (existingMetadata as any).isPlaceholder

  if (isPlaceholder) {
    console.log(`Replacing placeholder noun ${options.id} with real data`)
  }
}

3. Improved Placeholder Noun Handling

File: src/brainyData.ts (lines 2614, 2636)

• Added isPlaceholder: true flag to placeholder nouns created in addVerb() method
• Ensures placeholder nouns are marked as non-searchable mock data

const sourceMetadata = options.missingNounMetadata || {
  autoCreated: true,
  writeOnlyMode: true,
  isPlaceholder: true, // Mark as placeholder to exclude from search results
  // ... other metadata
}

4. Search Result Filtering

File: src/brainyData.ts (lines 1998-2006)

• Added filtering logic to exclude placeholder nouns from search results
• Prevents mock data from appearing in user-facing search results

// Filter out placeholder nouns from search results
searchResults = searchResults.filter(result => {
  if (result.metadata && typeof result.metadata === 'object') {
    const metadata = result.metadata as Record<string, any>
    return !metadata.isPlaceholder
  }
  return true
})

5. Enhanced Error Messages

File: src/brainyData.ts (lines 534-540)

• Updated checkWriteOnly() method to provide more helpful error messages
• Guides users to use get() for existence checks in write-only mode

private checkWriteOnly(allowExistenceChecks: boolean = false): void {
  if (this.writeOnly && !allowExistenceChecks) {
    throw new Error(
      'Cannot perform search operation: database is in write-only mode. Use get() for existence checks.'
    )
  }
}

Key Features Implemented

✅ Existence Checks in Write-Only Mode

• get() method now works in write-only mode by querying storage directly
• No need for separate writeOnlyMode parameter on addverb - the system auto-detects

✅ Placeholder Noun Management

• Placeholder nouns are marked with isPlaceholder: true flag
• Automatically filtered out of search results to prevent mock data visibility
• Mechanism to replace placeholders when real data is found

✅ Auto-Configuration

• Brainy automatically detects write-only mode and skips index loading
• Seamless fallback to storage queries when index is not available
• No additional configuration required from users

✅ Backward Compatibility

• All existing functionality preserved
• Enhanced error messages guide users to proper usage
• Graceful handling of edge cases and race conditions

Testing Results

The implementation was thoroughly tested with a comprehensive reproduction script that verified:

1. ✅ Search operations properly blocked in write-only mode with helpful error message
2. ✅ Existence checks (get operations) work in write-only mode via storage
3. ✅ Add operations can check for existing data in write-only mode
4. ✅ Placeholder nouns are filtered out of search results
5. ✅ Mechanism implemented to update placeholder nouns when real data is found
6. ✅ Auto-configuration: Brainy detects write-only mode and skips index loading

Impact

This implementation fully addresses the original GitHub issue requirements:

• Existence checks are no longer ignored in write-only mode
• They are performed directly against underlying storage as requested
• Placeholder nouns are properly handled and don't appear in search results
• Auto-configuration ensures the best user experience with minimal setup

The solution maintains the principle of making Brainy "auto configure and auto-tune itself so the user experience is simple as possible" while providing robust write-only mode functionality for high-performance data insertion scenarios.