fix: user metadata named 'level' is a real field everywhere — the engine-internal node layer no longer shadows it in sort/filter/aggregation, and the indexing views stop stamping a phantom 0 into its column; index epoch 2 rebuilds existing brains at first open

Also completes the v8.10.2 write-granularity law for the transact() plan
path: a metadata-only batch update never rewrites the vector-bearing noun
record (planUpdate staged the unconditional save the update() fix removed).
Seven pins in tests/integration/level-field-shadow.test.ts including the
reporting consumer's exact repro rows; orderBy JSDoc documents the ordering
contract and the announced field-addressing law.
This commit is contained in:
David Snelling 2026-08-03 11:57:32 -07:00
parent 099579f716
commit 958a085926
8 changed files with 259 additions and 17 deletions

View file

@ -55,6 +55,61 @@ Pairs with the native accelerator's same-day patch release; adopt as one bump.
---
## Unreleased (natural field names stop colliding with engine internals)
From a production report: sorting by a user metadata field named `level` silently
returned insertion order — the engine's internal HNSW node layer (also called
`level`) shadowed the user's field in every by-name read, and the indexing path
stamped a hardcoded `0` into the same index column (multi-valued poison). `level`
is a perfectly natural field name (game characters, priorities, floors); the
engine was wrong, not the caller.
- **`level` is user data now, everywhere.** Engine plumbing no longer resolves by
name, never shadows metadata, and never enters the indexed views. `orderBy:
'level'`, `where: { level: 9 }`, `groupBy: ['level']` all read YOUR field.
Regression pins: `tests/integration/level-field-shadow.test.ts` (the reporting
consumer's exact repro rows).
- **Index epoch 2.** The derived posting set changed, so every existing brain
rebuilds its metadata index from canonical at first open — poisoned columns
heal automatically; no manual step. First open after upgrade pays one rebuild
(observable via `getIndexStatus()`); pair this release with the same-day
native-accelerator release, which makes `level` indexable on the native path.
- **`transact()` metadata-only updates stop rewriting the vector record** — the
v8.10.2 write-granularity law now covers the batch/plan path too (it was
fixed for `update()` but the transact plan builder still staged the
unconditional save). If you batch stat touches through `transact()`, this is
your write-amplification fix.
- Coming next (announced so parsers and call sites can prepare): one
field-addressing law — bare names = user metadata, `system.<field>` for
engine fields, typed refusals for unresolvable names. Ships as its own
release with a migration advisory; nothing changes in this release.
---
## v8.10.2 — 2026-07-29 (metadata-only updates stop rewriting the vector record)
From a production incident on a large deployment: a read-heavy sweep that bumped
per-entity stats (metadata-only `update()` calls) saturated the disk — 5.8GB written
in 40 minutes — because every `update()` unconditionally re-persisted the WHOLE noun
record, unchanged vector included, fsynced.
- **`update()` write granularity fixed at the core.** A metadata-only update (no new
`data`, `vector`, or `type`) now writes the metadata leg and index deltas ONLY —
the vector-bearing noun record is never rewritten. Vector-side writes and HNSW
reindexing still happen exactly when the vector side actually changed. Regression
pins: `tests/integration/update-write-granularity.test.ts`.
- **Consumer guidance:** per-entity stat touches are now cheap, but batch them anyway
(one `transact()` instead of N `update()` calls) — granularity fixes the cost per
touch; batching fixes the count.
- Idle VFS `PathResolver` no longer logs `NaN% hit rate` once a minute (stats log
only on new traffic, at debug level).
- Native graph providers' `graph-lsm-*` storage keys are recognized as system
resources — the per-boot `Unknown key format` warning for them is gone.
Pairs with the native accelerator's same-day patch release; adopt as one bump.
---
## v8.10.1 — 2026-07-24 (the no-hot-retry contract + warm()'s metadata surface under native providers)
From a production incident: a native-provider op ground 38-40s inside a transaction,