fix(8.0): multi-valued array fields index every element (contains no longer misses)
addToIndex builds a fieldsMap from the extracted {field,value} entries, but
only __words__ accumulated — every other repeated field did fieldsMap[field]=value,
so a multi-valued field (tags:['a','b','c'] → three 'tags' entries from
extractIndexableFields) collapsed to last-value-wins. columnStore.addEntity
expands an array to one indexed entry per element, but it only ever received the
final scalar, so `contains` matched only the last element and missed the rest.
Accumulate any repeated non-__words__ field into an array before addEntity
(promote scalar→array on the second occurrence). __words__ keeps its always-array
special-case so its multiValue manifest flag stays set even for single-word docs.
find-unified-integration.test.ts → 0 failures (was 4):
- 'array contains' (A.4): now queries first/middle/last element — all match
(previously only the last-indexed element did).
- 'complete find workflow': $contains→contains (8.0 operator) AND fixed the test
premise — find() hard-ANDs vector∩graph∩metadata, and connected:{from:X} returns
X's NEIGHBOURS not X, so the graph anchor must be 'earth' (which the ML concept
relates to), not the concept itself. Multi-signal scores are reciprocal-rank
fusion (~1/61), not [0,1] cosine, so assert a positive fusion score, not >0.5.
- 'nonsense vector query': cosine search always returns nearest neighbours, so
assert the structural array/bound contract, not length===0 (a Tier-2 concern).
- 'hard-ANDs signals': a nonexistent connected.from is an empty graph signal that
zeros the intersection — assert length===0 (documented AND semantics), was >0.
1469 unit + full find-unified integration green.
This commit is contained in:
parent
009e50681e
commit
eccf42000b
2 changed files with 71 additions and 25 deletions
|
|
@ -1434,13 +1434,31 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
|
||||
// Write to column store — the single write path for all indexed data.
|
||||
// Converts extracted fields into a map and feeds the column store.
|
||||
//
|
||||
// extractIndexableFields emits ONE {field,value} entry per array element for
|
||||
// a multi-valued field (e.g. tags:['a','b','c'] → three 'tags' entries). We
|
||||
// must accumulate every repeated field into an array, not just __words__:
|
||||
// columnStore.addEntity expands an array value to one indexed entry per
|
||||
// element, so a scalar overwrite (last-value-wins) would index only the final
|
||||
// element and `contains` would miss the rest.
|
||||
if (this.columnStore) {
|
||||
const entityIntId = this.idMapper.getOrAssign(id)
|
||||
const fieldsMap: Record<string, unknown> = {}
|
||||
for (const { field, value } of fields) {
|
||||
if (field === '__words__') {
|
||||
// Always an array (keeps the field's multiValue manifest flag set even
|
||||
// for a single-word document).
|
||||
if (!fieldsMap.__words__) fieldsMap.__words__ = []
|
||||
;(fieldsMap.__words__ as number[]).push(value)
|
||||
;(fieldsMap.__words__ as unknown[]).push(value)
|
||||
} else if (field in fieldsMap) {
|
||||
// Repeated field → multi-valued. Promote the scalar to an array on the
|
||||
// second occurrence, then accumulate.
|
||||
const existing = fieldsMap[field]
|
||||
if (Array.isArray(existing)) {
|
||||
;(existing as unknown[]).push(value)
|
||||
} else {
|
||||
fieldsMap[field] = [existing, value]
|
||||
}
|
||||
} else {
|
||||
fieldsMap[field] = value
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue