fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1)

CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing
100% VFS file read failure. This fix restores functionality with production-grade
defense-in-depth architecture and comprehensive testing.

Problem:
- v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data
- Symptom: "Blob integrity check failed" on every VFS file read
- Impact: 100% failure rate in Workshop application
- Root Cause: Missing defense-in-depth unwrap verification

The Fix:
1. Defense-in-Depth Unwrapping
   - Added unwrap verification in BlobStorage.read() before hash check (line 342)
   - Added unwrap for metadata parsing (line 314)
   - Ensures data is always unwrapped regardless of adapter behavior

2. DRY Architecture
   - Created binaryDataCodec.ts as single source of truth
   - Refactored baseStorage to use shared utilities
   - All 8 storage adapters now use same implementation

3. Comprehensive Testing
   - Added TestWrappingAdapter that actually wraps like production
   - 3 new regression tests validate the fix
   - Tests exercise real wrapping scenario that caused the bug

Architecture Improvements:
-  Defense-in-Depth: Unwrap at BOTH adapter and blob layers
-  DRY Principle: Single source of truth in binaryDataCodec.ts
-  Works Across ALL Storage Adapters (8 total)
-  Prevents Future Regressions: Real wrapping tests

Files Changed:
- NEW: src/storage/cow/binaryDataCodec.ts (single source of truth)
- FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap)
- REFACTORED: src/storage/baseStorage.ts (uses shared codec)
- NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter)
- ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts
- UPDATED: CHANGELOG.md, package.json (v5.10.1)

Related Issues:
- v5.7.2: Original bug - hashed wrapper instead of content
- v5.7.5: First fix - added unwrap to adapter (necessary but insufficient)
- v5.10.0: Regression - missing defense-in-depth in BlobStorage
- v5.10.1: Complete fix - defense-in-depth + DRY + tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
David Snelling 2025-11-14 15:31:06 -08:00
parent 50ece5e179
commit 9a8b7a6cd4
8 changed files with 375 additions and 26 deletions

View file

@ -28,6 +28,7 @@ import { getShardIdFromUuid } from './sharding.js'
import { RefManager } from './cow/RefManager.js'
import { BlobStorage, type COWStorageAdapter } from './cow/BlobStorage.js'
import { CommitLog } from './cow/CommitLog.js'
import { unwrapBinaryData, wrapBinaryData } from './cow/binaryDataCodec.js'
import { prodLog } from '../utils/logger.js'
/**
@ -325,31 +326,19 @@ export abstract class BaseStorage extends BaseStorageAdapter {
if (data === null) {
return undefined
}
// Convert to Buffer
if (Buffer.isBuffer(data)) {
return data
}
// v5.7.5: Unwrap binary data stored as {_binary: true, data: "base64..."}
// v5.7.5/v5.10.1: Use shared binaryDataCodec utility (single source of truth)
// Unwraps binary data stored as {_binary: true, data: "base64..."}
// Fixes "Blob integrity check failed" - hash must be calculated on original content
if (data._binary && typeof data.data === 'string') {
return Buffer.from(data.data, 'base64')
}
return Buffer.from(JSON.stringify(data))
return unwrapBinaryData(data)
} catch (error) {
return undefined
}
},
put: async (key: string, data: Buffer): Promise<void> => {
// Store as Buffer (for blob data) or parse JSON (for metadata)
let obj: any
try {
// Try to parse as JSON first (for metadata)
obj = JSON.parse(data.toString())
} catch {
// Not JSON, store as binary (base64 encoded for JSON storage)
obj = { _binary: true, data: data.toString('base64') }
}
// v5.10.1: Use shared binaryDataCodec utility (single source of truth)
// Wraps binary data or parses JSON for storage
const obj = wrapBinaryData(data)
await this.writeObjectToPath(`_cow/${key}`, obj)
},