feat: array-unnest groupBy for aggregates + batch-embed entity extraction
- groupBy now supports { field, unnest: true }: an entity contributes once per
distinct element of an array field (tag frequency / faceted counts). Duplicate
elements on one entity count once; an empty/missing array joins no group. The
incremental add/update/delete paths fan out across the unnested groups.
- extractEntities/extractConcepts batch-embed the unique candidate spans in one
embedBatch call instead of one embed() per candidate (N sequential model calls);
falls back to per-candidate embedding if the batch fails. No behavior change.
This commit is contained in:
parent
2591001bd0
commit
c2e21b7b3c
8 changed files with 214 additions and 100 deletions
|
|
@ -151,6 +151,8 @@ export class EmbeddingSignal {
|
|||
definition?: string
|
||||
allTerms?: string[]
|
||||
metadata?: any
|
||||
/** Pre-computed candidate embedding (from a batch embed) — skips the per-candidate embed. */
|
||||
vector?: Vector
|
||||
}
|
||||
): Promise<TypeSignal | null> {
|
||||
this.stats.calls++
|
||||
|
|
@ -167,8 +169,8 @@ export class EmbeddingSignal {
|
|||
}
|
||||
|
||||
try {
|
||||
// Embed candidate once (efficiency!)
|
||||
const vector = await this.embedWithTimeout(candidate)
|
||||
// Use the pre-computed vector when the caller batch-embedded; otherwise embed once here.
|
||||
const vector = context?.vector ?? await this.embedWithTimeout(candidate)
|
||||
|
||||
// Check all three sources in parallel
|
||||
const [typeMatch, graphMatch, historyMatch] = await Promise.all([
|
||||
|
|
|
|||
Reference in a new issue