David Snelling
0c4a51c24e
7.x stored every object branch-scoped under branches/<branch>/<basePath>; 8.0 stores the IDENTICAL entity structure at the storage ROOT plus the generational _system/_generations layer. GenerationStore.open tolerates a missing manifest (opens at gen 0), so a naive 8.0 open of a 7.x directory reported ZERO entities and SILENTLY LOST ALL DATA. The fix is blocked in the normal init order — cor 3.0's storage-factory legacy guard fires inside setupStorage(), five steps before the old migration check — so a new phase runs BEFORE setupStorage(). legacyLayoutMigrationPhase() (init, between loadPlugins and setupStorage): - Fast-path: returns immediately when there's no top-level branches/ dir (every native 8.0 brain + fresh dir pays only one fs.existsSync on the init hot path). - Runs on a temporary BUILT-IN FileSystemStorage (never the plugin adapter, whose guard would throw). Memory/cloud/pre-built-adapter stores are a strict no-op. - Detect via _system/migration-layout.json marker + branches/<head>/entities/. - autoMigrate:false on a legacy layout THROWS explicit guidance (no silent loss). - Acquire the writer lock, then COLLAPSE branches/<head>/entities/* → entities/* via the .gz-transparent raw primitives (read→write→delete, idempotent/resume-safe; NOT fs.rename — logical paths + memory-adapter incompatibility). - Rebuild persisted counts (rebuildCounts → totalNounCount/counts.json; rebuildTypeCounts/rebuildSubtypeCounts → per-type stats) — the three indexes are rebuilt by the normal init's rebuildIndexesIfNeeded afterward. - Finalize: stamp the flat-v8 marker (re-open no-op), drain the head branch (non-head branches = 7.x version history 8.0's MVCC does not import; left as-is). Tests (tests/integration/migration-7x-to-8x.test.ts): the data-loss LOCK (a built-in FileSystemStorage on a reshaped 7.x dir reports 0 nouns), a byte-equal ROUND TRIP (find/related/counts equal the pre-reshape reference), IDEMPOTENCY, the autoMigrate:false GUARD throw, and the native-flat NO-OP. RELEASES upgrade checklist rewritten (the old note documented the opposite, lossy behavior). 1477 unit + migration 5 + db-mvcc 25 green; no init-path regression. Follow-ups (hardening, not correctness for the common case): backupTo config plumbing (currently a loud warning), a crash-mid-collapse resume test, and a boundary-safe fake-plugin ordering test. |
||
|---|---|---|
| .. | ||
| advanced-apis-regression.test.ts | ||
| aggregation.test.ts | ||
| all-apis-comprehensive.test.ts | ||
| api-parameter-validation.test.ts | ||
| batchImportWithRelations.test.ts | ||
| brainy-complete.integration.test.ts | ||
| brainy-core.integration.test.ts | ||
| brainy-phase1c-integration.test.ts | ||
| bun-compile-test.ts | ||
| clear-persistence.test.ts | ||
| clear-vfs.test.ts | ||
| cortex-compat.test.ts | ||
| count-synchronization.test.ts | ||
| db-mvcc.test.ts | ||
| db-temporal.test.ts | ||
| entity-confidence-weight.test.ts | ||
| find-limits.test.ts | ||
| find-triple-composition.test.ts | ||
| find-unified-integration.test.ts | ||
| find-where-zero.test.ts | ||
| get-relations-fix.test.ts | ||
| graphIndex-pagination.test.ts | ||
| hnsw-rebuild.test.ts | ||
| hybrid-search-vfs.test.ts | ||
| memory-enhancements-v5.11.0.test.ts | ||
| metadata-only-comprehensive.test.ts | ||
| metadata-vector-exclusion.test.ts | ||
| migration-7x-to-8x.test.ts | ||
| migration.test.ts | ||
| multi-process-safety.test.ts | ||
| orderby-sort-bug.test.ts | ||
| readAfterWrite.test.ts | ||
| relationship-intelligence.test.ts | ||
| remaining-apis.test.ts | ||
| rev-and-ifabsent.test.ts | ||
| savebinaryblob-concurrent-rename.test.ts | ||
| smart-import.test.ts | ||
| storage-batch-operations.test.ts | ||
| strict-mode-self-test.test.ts | ||
| subtype-and-facets.test.ts | ||
| update-asymmetry.test.ts | ||
| vector-recall.test.ts | ||
| verb-subtype-and-enforcement.test.ts | ||
| vfs-and-graph-entities.test.ts | ||
| vfs-api-wiring.test.ts | ||
| vfs-debug.test.ts | ||
| vfs-import-verification.test.ts | ||
| vfs-knowledge-separation.test.ts | ||
| vfs-performance-v5.11.1.test.ts | ||
| wasm-embeddings.test.ts | ||