feat: array-unnest groupBy for aggregates + batch-embed entity extraction
- groupBy now supports { field, unnest: true }: an entity contributes once per
distinct element of an array field (tag frequency / faceted counts). Duplicate
elements on one entity count once; an empty/missing array joins no group. The
incremental add/update/delete paths fan out across the unnested groups.
- extractEntities/extractConcepts batch-embed the unique candidate spans in one
embedBatch call instead of one embed() per candidate (N sequential model calls);
falls back to per-candidate embedding if the batch fails. No behavior change.
This commit is contained in:
parent
2591001bd0
commit
c2e21b7b3c
8 changed files with 214 additions and 100 deletions
|
|
@ -706,9 +706,17 @@ export type TimeWindowGranularity =
|
|||
| { seconds: number }
|
||||
|
||||
/**
|
||||
* A GROUP BY dimension — either a plain metadata field or a time-windowed field
|
||||
* A GROUP BY dimension — one of:
|
||||
* - a plain metadata field name (string),
|
||||
* - a time-windowed field (`{ field, window }`), or
|
||||
* - an **unnest** field (`{ field, unnest: true }`): the field holds an array, and the entity
|
||||
* contributes once to a group per distinct element (e.g. `tags: string[]` → tag frequency).
|
||||
* An entity whose unnest field is missing or an empty array contributes to no group.
|
||||
*/
|
||||
export type GroupByDimension = string | { field: string; window: TimeWindowGranularity }
|
||||
export type GroupByDimension =
|
||||
| string
|
||||
| { field: string; window: TimeWindowGranularity }
|
||||
| { field: string; unnest: true }
|
||||
|
||||
/**
|
||||
* Source filter for which entities feed into an aggregate
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue