open-brainy/src/storage/cow/binaryDataCodec.ts

123 lines
3.2 KiB
TypeScript
Raw Normal View History

fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
/**
* Binary Data Codec: Single Source of Truth for Wrap/Unwrap Operations
*
* This module provides the ONLY implementation of binary data encoding/decoding
* used across all storage adapters and blob storage.
*
* Design Principles:
* - DRY: One implementation, used everywhere
* - Single Responsibility: Only handles binary JSON conversion
* - Type-Safe: Proper TypeScript types
* - Defensive: Handles all edge cases
*
* Used by:
* - BaseStorage COW adapter (write/read operations)
* - BlobStorage (defense-in-depth verification)
* - All storage adapters (via BaseStorage)
*
* @module storage/cow/binaryDataCodec
*/
/**
* Wrapped binary data format
* Used when storing binary data in JSON-based storage
*/
export interface WrappedBinaryData {
_binary: true
data: string // base64-encoded
}
/**
* Check if data is wrapped binary format
*/
export function isWrappedBinary(data: any): data is WrappedBinaryData {
return (
typeof data === 'object' &&
data !== null &&
data._binary === true &&
typeof data.data === 'string'
)
}
/**
* Unwrap binary data from JSON wrapper
*
* This is the SINGLE SOURCE OF TRUTH for unwrapping binary data.
* All storage operations MUST use this function.
*
* Handles:
* - Buffer Buffer (pass-through)
* - {_binary: true, data: "base64..."} Buffer (unwrap)
* - Plain object Buffer (JSON stringify)
* - Other types Error
*
* @param data - Data to unwrap (may be Buffer, wrapped object, or plain object)
* @returns Unwrapped Buffer
* @throws Error if data type is invalid
*/
export function unwrapBinaryData(data: any): Buffer {
// Case 1: Already a Buffer (no unwrapping needed)
if (Buffer.isBuffer(data)) {
return data
}
// Case 2: Wrapped binary data {_binary: true, data: "base64..."}
if (isWrappedBinary(data)) {
return Buffer.from(data.data, 'base64')
}
// Case 3: Plain object (shouldn't happen for binary blobs, but handle gracefully)
if (typeof data === 'object' && data !== null) {
return Buffer.from(JSON.stringify(data))
}
// Case 4: String (convert to Buffer)
if (typeof data === 'string') {
return Buffer.from(data)
}
// Case 5: Invalid type
throw new Error(
`Invalid data type for unwrap: ${typeof data}. ` +
`Expected Buffer or {_binary: true, data: "base64..."}`
)
}
/**
* Wrap binary data for JSON storage
*
* This is the SINGLE SOURCE OF TRUTH for wrapping binary data.
* All storage operations MUST use this function.
*
* @param data - Buffer to wrap
* @returns Wrapped object or parsed JSON object
*/
export function wrapBinaryData(data: Buffer): any {
// Try to parse as JSON first (for metadata, trees, commits)
try {
return JSON.parse(data.toString())
} catch {
// Not JSON - wrap as binary data
return {
_binary: true,
data: data.toString('base64')
} as WrappedBinaryData
}
}
/**
* Ensure data is a Buffer
*
* Convenience function that combines type checking and unwrapping.
* Use this when you need to ensure you have a Buffer.
*
* @param data - Data that should be or can be converted to Buffer
* @returns Buffer
*/
export function ensureBuffer(data: any): Buffer {
if (Buffer.isBuffer(data)) {
return data
}
return unwrapBinaryData(data)
}