David Snelling
24bf6cdbc5
The write side of the law, ruled 2026-08-03: data is either in main space where developers can use anything, or it is in system.*. - The reserved-name write door DIES: add/update/relate/updateRelation metadata bags accept EVERY name (confidence, type, id, data, level, content, ...) as ordinary user fields — indexed, filterable, sortable, aggregatable, identical to any other field. The remap/enforce/warn machinery, the reservedFieldPolicy config (now a typed init refusal), and the compile-time metadata key bans are all removed. The one write refusal left: keys spelled 'system.*' (namespace forgery), now enforced on all four write doors. - STORED RECORDS GO NESTED (v2): engine fields top-level, the user bag nested verbatim under 'metadata', sealed by a format stamp — by-name storage discrimination is unsound once colliders are admitted. Legacy flat records stay readable forever through the shape-aware splitters (sound for them: the old door refused colliders). Time travel rides the same split (generation store snapshots whole records). - Name-based index exclusions DIE: user frame indexes every name; the excludeFields/indexedFields knobs and their silent-[] holes are gone; bulk-payload protection is value-shape only, uniform across names. - Consumer-sweep findings fixed in the same wave: per-type counts read the frozen 'system.type' column (addToIndex sort, affinity tracking, cold-count rehydration, VFS type bitmaps — legacy 'noun' fallback for pre-rebuild reads); resolveHiddenIds addresses 'system.visibility' (bare 'visibility' was a silent no-op under the law — VFS/system entities leaked into default reads). - Fidelity fallout fixed in the owning layers: readEntityFieldAddress reads the bag first (colliders were absent-shadowed by its own guard) and never serves system addresses from the bag; blob history refs read the bag shape-aware; migration transforms now receive ONE normalized view (engine fields + nested bag) regardless of stored era, and stray flat-habit keys refuse with the fix in the message. - THE REOPEN-COLLIDER CONFORMANCE CASE (required before any RC counts as gates-green): all ten collider names + plumbing names written as user fields, verified verbatim + queryable across live reads, flush+reopen, a forced epoch rebuild, and asOf time travel; relation mirror; forgery refusals; legacy flat-record compat. 8/8 green. Gates: unit 1901/1901 (exit 0) · integration 758 (exit 0) · conformance 27/27 (exit 0) · consumer test sweep migrated (10 files).
61 lines
2.6 KiB
TypeScript
61 lines
2.6 KiB
TypeScript
/**
|
|
* @module tests/unit/brainy/find-orderby-pagek
|
|
* @description CTX-BR-FIND-ORDERBY item (1) — `find({ where, orderBy, limit })` must
|
|
* produce only the requested PAGE of sorted ids, not the full sorted match set. At
|
|
* billion scale a broad filter + orderBy returning 20 rows previously materialized
|
|
* every matching sorted id (O(matches) heap); the page bound (`offset+limit`) is now
|
|
* threaded into `getSortedIdsForFilter` → the column store's top-K heap. Ordering
|
|
* correctness is covered by orderby-sort-bug.test.ts; this locks the page bound.
|
|
*/
|
|
|
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
|
import { Brainy } from '../../../src/index.js'
|
|
import { NounType } from '../../../src/types/graphTypes.js'
|
|
import { createTestConfig } from '../../helpers/test-factory.js'
|
|
|
|
describe('find({ where, orderBy }) bounds the sort to the page (CTX-BR-FIND-ORDERBY #1)', () => {
|
|
let brain: Brainy
|
|
const MATCHES = 50
|
|
|
|
beforeEach(async () => {
|
|
brain = new Brainy(createTestConfig())
|
|
await brain.init()
|
|
for (let i = 0; i < MATCHES; i++) {
|
|
await brain.add({ type: NounType.Document, data: `doc ${i}`, metadata: { bucket: 'x', seq: i } })
|
|
}
|
|
})
|
|
|
|
afterEach(async () => {
|
|
await brain.close()
|
|
})
|
|
|
|
it('threads the page bound (offset+limit) into getSortedIdsForFilter, not the full match count', async () => {
|
|
let capturedTopK: number | undefined
|
|
const real = (brain as any).metadataIndex.getSortedIdsForFilter.bind((brain as any).metadataIndex)
|
|
;(brain as any).metadataIndex.getSortedIdsForFilter = async (
|
|
f: any,
|
|
ob: string,
|
|
o: 'asc' | 'desc',
|
|
topK?: number
|
|
) => {
|
|
capturedTopK = topK
|
|
return real(f, ob, o, topK)
|
|
}
|
|
|
|
// system.createdAt — entity age, not a user metadata field named 'createdAt'.
|
|
const results = await brain.find({ where: { bucket: 'x' }, orderBy: 'system.createdAt', order: 'desc', limit: 5 })
|
|
|
|
expect(results).toHaveLength(5)
|
|
// Page-bounded: ~ limit (5) + a small hidden-tier over-fetch — NOT all 50 matches.
|
|
expect(capturedTopK).toBeDefined()
|
|
expect(capturedTopK!).toBeGreaterThanOrEqual(5)
|
|
expect(capturedTopK!).toBeLessThan(MATCHES)
|
|
})
|
|
|
|
it('still returns the correct page with offset (sort + window unchanged)', async () => {
|
|
// seq is a numeric metadata field; orderBy seq asc → 0,1,2,… ; page [10,15).
|
|
const page = await brain.find({ where: { bucket: 'x' }, orderBy: 'seq', order: 'asc', limit: 5, offset: 10 })
|
|
expect(page).toHaveLength(5)
|
|
expect(page.map((r) => (r.metadata as { seq: number }).seq)).toEqual([10, 11, 12, 13, 14])
|
|
})
|
|
})
|