feat(query): the sparse-store cut — where on a never-carried field serves operator truth, never a refusal
A first adopter's namespace migration went 341 red on one class: the never-carried-field refusal firing on CORRECT filters against fresh and sparse stores — a freshly provisioned tenant refused its own first filtered read, with the did-you-mean built for typos firing hardest on day-one stores where nothing is wrong. The ruled cut: a WHERE filter naming a field no row carries is SERVED OPERATOR-TRUTHFULLY — eq/in/range/contains answer [] (nothing carries it, nothing matches); ne and exists:false answer ALL rows (the equally true complement — a blanket empty here would be silently wrong, which is why the simpler cut was rejected); exists:true answers []. Served from the field registry, with the did-you-mean demoted to a once-per-field WARN. orderBy and genuinely ambiguous addresses KEEP their hard typed refusals: no truthful order exists over an uncarried field, and ambiguity is a contract error while absence is data. Mechanics: the negative operator absorbs the FIELD_NOT_INDEXED throw as its empty exclude set (the clause-level catch correctly zeroes positive operators only); the egress matcher already agreed. Plus the provider-seam belt: a field refusal thrown by a replacement metadata manager is normalized to THIS package's UnresolvableFieldError at every filter call site — one class identity for consumers, instanceof works (a first adopter's cross-package finding). Conformance: tests/conformance/sparse-store-cut.test.ts — the shared operator rows both engines run (positive-empty, negative-all, fresh-tenant day-one, orderBy refusal kept, compound composition). Gates: unit 2065/2065 · integration 832 · conformance 36/36.
This commit is contained in:
parent
df96fccfd1
commit
7b67db4d0c
5 changed files with 188 additions and 8 deletions
|
|
@ -1908,6 +1908,35 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
* index (early-stop at `offset+limit`); the JS index returns ALL matches and lets
|
||||
* the caller window them, so `_opts` is intentionally ignored here.
|
||||
*/
|
||||
/** Once-per-field throttle for the sparse-store did-you-mean WARN. */
|
||||
private readonly warnedNeverCarried = new Set<string>()
|
||||
|
||||
/**
|
||||
* THE SPARSE-STORE CUT (ruled 2026-08-12): a WHERE filter naming a field
|
||||
* no row carries is SERVED OPERATOR-TRUTHFULLY (eq/range/contains → [];
|
||||
* ne/exists:false → all rows; exists:true → []) — the JS evaluator below
|
||||
* already computes exactly these truths via complements — with the
|
||||
* did-you-mean demoted to this throttled WARN. A fresh store's first
|
||||
* filtered read is a correct empty answer, never a refusal. orderBy and
|
||||
* ambiguous addresses KEEP their hard refusals (no truthful order
|
||||
* exists; ambiguity is a contract error — absence is data).
|
||||
*/
|
||||
/** Is this field known to the index at all (any row ever carried it)? */
|
||||
private fieldRegistryHas(field: string): boolean {
|
||||
return this.fieldStats.has(field)
|
||||
}
|
||||
|
||||
private warnNeverCarriedOnce(field: string): void {
|
||||
if (this.warnedNeverCarried.has(field)) return
|
||||
this.warnedNeverCarried.add(field)
|
||||
prodLog.warn(
|
||||
`[MetadataIndex] filter names field '${field}' which no row carries — ` +
|
||||
`serving the operator-truthful answer (empty for positive matches; ` +
|
||||
`the complement for ne/exists:false). If this is a typo, check the ` +
|
||||
`field name; refusals remain on orderBy.`
|
||||
)
|
||||
}
|
||||
|
||||
async getIdsForFilter(filter: any, _opts?: { limit?: number; offset?: number }): Promise<string[]> {
|
||||
if (!filter || Object.keys(filter).length === 0) {
|
||||
return []
|
||||
|
|
@ -1984,6 +2013,17 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
const address = parseFieldAddress(rawField, 'entity')
|
||||
const field = address.scope === 'system' ? `system.${address.field}` : address.field
|
||||
|
||||
// Sparse-store cut: a user field no row carries serves operator-truth
|
||||
// below (the evaluators' complements are already correct) — announce
|
||||
// it once so a typo is findable without breaking a fresh store.
|
||||
if (
|
||||
address.scope !== 'system' &&
|
||||
!(this.columnStore && this.columnStore.hasField(field)) &&
|
||||
!this.fieldRegistryHas(field)
|
||||
) {
|
||||
this.warnNeverCarriedOnce(field)
|
||||
}
|
||||
|
||||
let fieldResults: string[] = []
|
||||
|
||||
try {
|
||||
|
|
@ -2022,7 +2062,18 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
// complement as a bitmap difference over the int-id universe rather
|
||||
// than materializing the whole corpus as UUID strings to filter it.
|
||||
const excludeInts: number[] = []
|
||||
for (const uuid of await this.getIds(field, operand)) {
|
||||
// Sparse-store truth: a never-carried field has NOTHING to
|
||||
// exclude — the complement of nothing is EVERYTHING. getIds
|
||||
// throws FIELD_NOT_INDEXED there; the clause-level catch
|
||||
// would wrongly zero this NEGATIVE operator, so absorb it
|
||||
// here as the empty exclude set (the ruled operator-truth).
|
||||
let neMatches: string[] = []
|
||||
try {
|
||||
neMatches = await this.getIds(field, operand)
|
||||
} catch {
|
||||
neMatches = []
|
||||
}
|
||||
for (const uuid of neMatches) {
|
||||
const intId = this.idMapper.getInt(uuid)
|
||||
if (intId !== undefined) excludeInts.push(intId)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue