- groupBy now supports { field, unnest: true }: an entity contributes once per
distinct element of an array field (tag frequency / faceted counts). Duplicate
elements on one entity count once; an empty/missing array joins no group. The
incremental add/update/delete paths fan out across the unnested groups.
- extractEntities/extractConcepts batch-embed the unique candidate spans in one
embedBatch call instead of one embed() per candidate (N sequential model calls);
falls back to per-candidate embedding if the batch fails. No behavior change.
New report APIs:
- brain.queryAggregate(name, params) returns the clean AggregateResult[] shape
({ groupKey, metrics, count }) directly, instead of the search-Result wrapper that
find({ aggregate }) returns.
- HAVING: find({ aggregate, having }) and queryAggregate filter groups by computed
metric values (e.g. revenue > 1000), complementing where (which filters group keys).
Evaluated per group: O(groups), independent of entity count.
Fixes (all reproducible on Node; surfaced under Cortex+Bun by Memory):
- Aggregate backfill-on-define: defining an aggregate over a store that already holds
matching entities returned []. It now backfills from existing entities on first query
(storage-agnostic via getNouns), so it works under durable backends that reopen
pre-populated. groupBy:['noun'] resolves to the entity type; find({ aggregate }) rows
expose groupKey/metrics/count at the top level.
- Multi-hop find({ connected: { depth, via } }) honors depth and via at every hop
(was 1-hop only, with verb filtering applied to hop 1 only); BFS bounded by limit.
- extractEntities no longer bleeds a neighbour's type indicator across candidates; each
candidate is typed by its own span. Also fixes the "Dr." title pattern.
Adds real-embedding regression tests; full unit suite green.
Preventive cleanup of sites that previously used dual-lookup patterns
(metadata[field] ?? entity[field]) or hardcoded timestamp if-chains
to read fields off HNSWNounWithMetadata. These worked today but were
fragile — the same failure mode that caused the orderBy sort bug
would have recurred the next time someone reordered or dropped a
lookup.
- AggregationIndex.computeGroupKey + getNumericField now route all
dimension and metric field lookups through resolveEntityField.
- improvedNeuralAPI._getItemsByTimeWindow + _clusterItemsByField
use resolveEntityField as the primary path, with item.data as a
legacy producer fallback. The type/nounType legacy branch is
preserved as-is since it handles storage format compatibility
rather than shape contract drift.
No behavior change for correct inputs. All aggregation and orderby
regression tests pass unchanged.
Add a write-time incremental aggregation engine that maintains running
totals on every add/update/delete for O(1) read performance. Integrates
into brain.find({ aggregate }) for a unified query API.
Core features:
- AggregationIndex with defineAggregate()/removeAggregate() API
- Five aggregation operations: SUM, COUNT, AVG, MIN, MAX
- GROUP BY with multiple dimensions including time windows
- Time window bucketing: hour, day, week, month, quarter, year, custom
- Materialization of results as NounType.Measurement entities
- Debounced persistence of definitions and state to storage
- Definition change detection via FNV-1a hashing with auto-rebuild
- Infinite loop prevention for materialized entities
- 'aggregation' plugin provider key for native acceleration
- Lazy initialization (created on first defineAggregate() call)
- 73 tests (unit + integration) covering all functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
