soulcraft/brainy
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: array-unnest groupBy for aggregates + batch-embed entity extraction

Browse source 
- groupBy now supports { field, unnest: true }: an entity contributes once per
  distinct element of an array field (tag frequency / faceted counts). Duplicate
  elements on one entity count once; an empty/missing array joins no group. The
  incremental add/update/delete paths fan out across the unnested groups.
- extractEntities/extractConcepts batch-embed the unique candidate spans in one
  embedBatch call instead of one embed() per candidate (N sequential model calls);
  falls back to per-candidate embedding if the batch fails. No behavior change.
This commit is contained in:
David Snelling 2026-05-26 14:20:40 -07:00
parent 2591001bd0
commit c2e21b7b3c
8 changed files with 214 additions and 100 deletions
Download patch file Download diff file Expand all files Collapse all files

19
docs/guides/aggregation.md
View file

@ -200,6 +200,25 @@ brain.defineAggregate({
// { region: 'EU', date: '2024-01' }
```
### Array Fields (Unnest)
Group by **each element** of an array-valued field — for tag frequencies, label counts, and
faceted breakdowns. Mark the dimension `{ field, unnest: true }`:
```typescript
brain.defineAggregate({
name: 'tag_frequency',
source: { type: NounType.Document },
groupBy: [{ field: 'tags', unnest: true }],
metrics: { count: { op: 'count' } }
})
// A document tagged ['ml', 'ai'] contributes once to the 'ml' group and once to 'ai'.
// Duplicate tags on one entity count once; an entity with no tags joins no group.
const top = await brain.queryAggregate('tag_frequency', { orderBy: 'count', order: 'desc' })
// [ { groupKey: { tags: 'ai' }, metrics: { count: 3 }, count: 3 }, ... ]
```
## Querying Aggregates
Aggregate results are queried through the standard `find()` method.