brainy/tests/helpers/TestWrappingAdapter.ts

86 lines
2.4 KiB
TypeScript
Raw Permalink Normal View History

fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
/**
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API The COW version-control surface (fork, branches, checkout, commit, getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with its subsystems: src/versioning/, the COW object store (CommitLog, CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/ with/persist/restore) is the one versioning model in 8.0. Survivors and replacements: - BlobStorage survives (the VFS stores file content through it), relocated to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter interface is now BlobStoreAdapter, slimmed to the consumed surface (write/read/has/delete/getMetadata + MIME-aware compression policy). - brain.migrate() backup branches are replaced by persist-before-migrate: MigrateOptions.backupTo persists a hard-link snapshot of the current generation before any transform runs; MigrationResult.backupPath reports it, and brain.restore(path) brings it back wholesale. - CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by snapshot.ts — snapshot <path>, restore <path>, history (tx-log), generation. - New public read API: brain.transactionLog({limit}) exposes the reified tx-log (generation/timestamp/meta, newest first) that backs the CLI history command; TxLogEntry is exported. Tests: superseded suites deleted; fork/commit blocks excised from shared suites; BlobStorage tests relocated + reworked against the slimmed store; migration tests now prove the backupTo snapshot/restore round trip; new transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
* TestWrappingAdapter: Real blob store adapter for testing
fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
*
* This adapter ACTUALLY wraps binary data in JSON (like production storage)
* to properly test the unwrap logic in BlobStorage.
*
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API The COW version-control surface (fork, branches, checkout, commit, getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with its subsystems: src/versioning/, the COW object store (CommitLog, CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/ with/persist/restore) is the one versioning model in 8.0. Survivors and replacements: - BlobStorage survives (the VFS stores file content through it), relocated to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter interface is now BlobStoreAdapter, slimmed to the consumed surface (write/read/has/delete/getMetadata + MIME-aware compression policy). - brain.migrate() backup branches are replaced by persist-before-migrate: MigrateOptions.backupTo persists a hard-link snapshot of the current generation before any transform runs; MigrationResult.backupPath reports it, and brain.restore(path) brings it back wholesale. - CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by snapshot.ts — snapshot <path>, restore <path>, history (tx-log), generation. - New public read API: brain.transactionLog({limit}) exposes the reified tx-log (generation/timestamp/meta, newest first) that backs the CLI history command; TxLogEntry is exported. Tests: superseded suites deleted; fork/commit blocks excised from shared suites; BlobStorage tests relocated + reworked against the slimmed store; migration tests now prove the backupTo snapshot/restore round trip; new transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
* Unlike InMemoryBlobAdapter which stores Buffers directly,
fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
* this adapter simulates real storage behavior:
* 1. Compresses with gzip
* 2. Wraps binary as {_binary: true, data: "base64..."}
* 3. Parses JSON on read (returns JS object, not Buffer)
*
* This ensures tests exercise the ACTUAL code path that caused v5.10.0 regression.
*/
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API The COW version-control surface (fork, branches, checkout, commit, getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with its subsystems: src/versioning/, the COW object store (CommitLog, CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/ with/persist/restore) is the one versioning model in 8.0. Survivors and replacements: - BlobStorage survives (the VFS stores file content through it), relocated to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter interface is now BlobStoreAdapter, slimmed to the consumed surface (write/read/has/delete/getMetadata + MIME-aware compression policy). - brain.migrate() backup branches are replaced by persist-before-migrate: MigrateOptions.backupTo persists a hard-link snapshot of the current generation before any transform runs; MigrationResult.backupPath reports it, and brain.restore(path) brings it back wholesale. - CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by snapshot.ts — snapshot <path>, restore <path>, history (tx-log), generation. - New public read API: brain.transactionLog({limit}) exposes the reified tx-log (generation/timestamp/meta, newest first) that backs the CLI history command; TxLogEntry is exported. Tests: superseded suites deleted; fork/commit blocks excised from shared suites; BlobStorage tests relocated + reworked against the slimmed store; migration tests now prove the backupTo snapshot/restore round trip; new transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
import { BlobStoreAdapter } from '../../src/storage/blobStorage.js'
fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
import { gzip, gunzip } from 'zlib'
import { promisify } from 'util'
const gzipAsync = promisify(gzip)
const gunzipAsync = promisify(gunzip)
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API The COW version-control surface (fork, branches, checkout, commit, getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with its subsystems: src/versioning/, the COW object store (CommitLog, CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/ with/persist/restore) is the one versioning model in 8.0. Survivors and replacements: - BlobStorage survives (the VFS stores file content through it), relocated to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter interface is now BlobStoreAdapter, slimmed to the consumed surface (write/read/has/delete/getMetadata + MIME-aware compression policy). - brain.migrate() backup branches are replaced by persist-before-migrate: MigrateOptions.backupTo persists a hard-link snapshot of the current generation before any transform runs; MigrationResult.backupPath reports it, and brain.restore(path) brings it back wholesale. - CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by snapshot.ts — snapshot <path>, restore <path>, history (tx-log), generation. - New public read API: brain.transactionLog({limit}) exposes the reified tx-log (generation/timestamp/meta, newest first) that backs the CLI history command; TxLogEntry is exported. Tests: superseded suites deleted; fork/commit blocks excised from shared suites; BlobStorage tests relocated + reworked against the slimmed store; migration tests now prove the backupTo snapshot/restore round trip; new transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
export class TestWrappingAdapter implements BlobStoreAdapter {
fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
private storage = new Map<string, Buffer>()
async get(key: string): Promise<any | undefined> {
const compressed = this.storage.get(key)
if (!compressed) {
return undefined
}
// Decompress (like real storage)
const decompressed = await gunzipAsync(compressed)
// Parse JSON (like real storage)
// This returns a JS object: {_binary: true, data: "base64..."}
// NOT a Buffer!
return JSON.parse(decompressed.toString())
}
async put(key: string, data: Buffer): Promise<void> {
feat(8.0)!: delete fork/branch/commit/history/versions — superseded by the Db API The COW version-control surface (fork, branches, checkout, commit, getHistory/streamHistory, asOfCommit, brain.versions) is gone, along with its subsystems: src/versioning/, the COW object store (CommitLog, CommitObject, RefManager, TreeObject), HistoricalStorageAdapter, and the TypeAwareHNSWIndex path it kept alive. The Db API (now/transact/asOf/ with/persist/restore) is the one versioning model in 8.0. Survivors and replacements: - BlobStorage survives (the VFS stores file content through it), relocated to src/storage/blobStorage.ts with binaryDataCodec.ts; its adapter interface is now BlobStoreAdapter, slimmed to the consumed surface (write/read/has/delete/getMetadata + MIME-aware compression policy). - brain.migrate() backup branches are replaced by persist-before-migrate: MigrateOptions.backupTo persists a hard-link snapshot of the current generation before any transform runs; MigrationResult.backupPath reports it, and brain.restore(path) brings it back wholesale. - CLI: cow.ts (fork/branch/checkout/history/migrate) is replaced by snapshot.ts — snapshot <path>, restore <path>, history (tx-log), generation. - New public read API: brain.transactionLog({limit}) exposes the reified tx-log (generation/timestamp/meta, newest first) that backs the CLI history command; TxLogEntry is exported. Tests: superseded suites deleted; fork/commit blocks excised from shared suites; BlobStorage tests relocated + reworked against the slimmed store; migration tests now prove the backupTo snapshot/restore round trip; new transactionLog coverage in db-mvcc.
2026-06-10 15:22:47 -07:00
// Use key-based dispatch (matches the baseStorage blob adapter)
perf: eliminate N+1 patterns across all APIs for 10-20x faster cloud storage Fixed 8 N+1 patterns that caused severe performance degradation on cloud storage (GCS, S3, Azure, R2): **Core Issues Fixed:** - find(): 5 code paths loaded entities one-by-one (10x slower) - batchGet() with vectors: Looped individual get() calls (10x slower) - executeGraphSearch(): Loaded connected entities individually (20x slower) - relate() duplicate check: Loaded relationships one-by-one (5x slower) - deleteMany(): Separate transaction per entity (10x slower) - VFS tree loading: N+1 getChildren() calls (53x slower) - VFS file operations: updateAccessTime() write on every read (2-3x slower) **Solutions Implemented:** 1. Batch entity loading in find() - 5 locations - Replace individual get() with batchGet() - GCS: 10 entities = 500ms → 50ms (10x faster) 2. Added storage.getNounBatch(ids) method - Batch-loads vectors + metadata in parallel - Eliminates N+1 for includeVectors: true 3. Added storage.getVerbsBatch(ids) method - Batch-loads relationships with metadata - Used by relate() duplicate checking 4. Added graphIndex.getVerbsBatchCached(ids) - Cache-aware batch verb loading - Checks UnifiedCache before storage 5. Optimized deleteMany() with transaction batching - Chunks of 10 entities per transaction - Atomic within chunk, graceful across chunks 6. Fixed VFS tree traversal N+1 pattern - Graph traversal + ONE batch fetch - 111 calls → 1 call (111x reduction) 7. Removed VFS updateAccessTime() on reads - Eliminated 50-100ms write per read - Follows modern filesystem noatime practice **Performance Impact (Production GCS):** | Operation | Before | After | Speedup | |-----------|--------|-------|---------| | find() 10 results | 500ms | 50ms | 10x | | batchGet() 10 vectors | 500ms | 50ms | 10x | | executeGraphSearch() 20 | 1000ms | 50ms | 20x | | relate() duplicate (5) | 250ms | 50ms | 5x | | deleteMany() 10 entities | 2000ms | 200ms | 10x | | VFS tree loading | 5304ms | 100ms | 53x | | VFS readFile() | 100-150ms | 50ms | 2-3x | **Architecture:** - All batch methods use readBatchWithInheritance() for COW/fork/asOf support - Works with all storage adapters (GCS, S3, Azure, R2, OPFS, FileSystem) - Cache-aware with proper UnifiedCache integration - Transaction-safe with atomic chunked operations - Fully backward compatible **Files Modified:** - src/brainy.ts: Fixed find(), batchGet(), relate(), deleteMany(), executeGraphSearch() - src/storage/baseStorage.ts: Added getNounBatch(), getVerbsBatch() - src/graph/graphAdjacencyIndex.ts: Added getVerbsBatchCached() - src/vfs/VirtualFileSystem.ts: Fixed tree traversal, removed updateAccessTime() - src/coreTypes.ts: Added batch method signatures to StorageAdapter - src/types/brainy.types.ts: Added continueOnError to DeleteManyParams - tests/: Added comprehensive regression tests **Overall Impact:** - 10-20x faster batch operations on cloud storage - 50-90% cost reduction (fewer storage API calls) - Production-ready with clean architecture - Zero breaking changes - automatic performance improvement 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 15:18:26 -08:00
// NO GUESSING - key format explicitly declares data type
const obj = key.includes('-meta:') || key.startsWith('ref:')
? JSON.parse(data.toString()) // Metadata/refs: ALWAYS JSON
: { _binary: true, data: data.toString('base64') } // Blobs: ALWAYS binary
fix: critical blob integrity regression with defense-in-depth architecture (v5.10.1) CRITICAL BUG FIX: v5.10.0 regressed the v5.7.2 blob integrity bug, causing 100% VFS file read failure. This fix restores functionality with production-grade defense-in-depth architecture and comprehensive testing. Problem: - v5.10.0 reintroduced bug where BlobStorage.read() hashed wrapped data - Symptom: "Blob integrity check failed" on every VFS file read - Impact: 100% failure rate in Workshop application - Root Cause: Missing defense-in-depth unwrap verification The Fix: 1. Defense-in-Depth Unwrapping - Added unwrap verification in BlobStorage.read() before hash check (line 342) - Added unwrap for metadata parsing (line 314) - Ensures data is always unwrapped regardless of adapter behavior 2. DRY Architecture - Created binaryDataCodec.ts as single source of truth - Refactored baseStorage to use shared utilities - All 8 storage adapters now use same implementation 3. Comprehensive Testing - Added TestWrappingAdapter that actually wraps like production - 3 new regression tests validate the fix - Tests exercise real wrapping scenario that caused the bug Architecture Improvements: - ✅ Defense-in-Depth: Unwrap at BOTH adapter and blob layers - ✅ DRY Principle: Single source of truth in binaryDataCodec.ts - ✅ Works Across ALL Storage Adapters (8 total) - ✅ Prevents Future Regressions: Real wrapping tests Files Changed: - NEW: src/storage/cow/binaryDataCodec.ts (single source of truth) - FIXED: src/storage/cow/BlobStorage.ts (defense-in-depth unwrap) - REFACTORED: src/storage/baseStorage.ts (uses shared codec) - NEW: tests/helpers/TestWrappingAdapter.ts (real wrapping adapter) - ADDED: 3 regression tests in tests/unit/storage/cow/BlobStorage.test.ts - UPDATED: CHANGELOG.md, package.json (v5.10.1) Related Issues: - v5.7.2: Original bug - hashed wrapper instead of content - v5.7.5: First fix - added unwrap to adapter (necessary but insufficient) - v5.10.0: Regression - missing defense-in-depth in BlobStorage - v5.10.1: Complete fix - defense-in-depth + DRY + tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 15:31:06 -08:00
// Stringify to JSON
const jsonStr = JSON.stringify(obj)
// Compress (like real storage)
const compressed = await gzipAsync(Buffer.from(jsonStr))
// Store compressed data
this.storage.set(key, compressed)
}
async delete(key: string): Promise<void> {
this.storage.delete(key)
}
async list(prefix: string): Promise<string[]> {
const keys: string[] = []
for (const key of this.storage.keys()) {
if (key.startsWith(prefix)) {
keys.push(key)
}
}
return keys
}
/**
* Clear all storage (for test cleanup)
*/
clear(): void {
this.storage.clear()
}
/**
* Get storage size (for testing)
*/
size(): number {
return this.storage.size
}
}