brainy/docs/QUERY_OPERATORS.md

268 lines
7.3 KiB
Markdown
Raw Normal View History

# Query Operators (BFO)
> Brainy Field Operators — the complete reference for `where` filters in `find()`.
All operators work with `find({ where: { ... } })` and filter on **metadata fields** (not `data`).
---
## Equality
| Operator | Alias | Description | Example |
|----------|-------|-------------|---------|
| `equals` | `eq`, `is` | Exact match | `{ status: { equals: 'active' } }` |
| `notEquals` | `ne`, `isNot` | Not equal | `{ status: { notEquals: 'deleted' } }` |
**Shorthand:** A bare value is treated as `equals`:
```typescript
// These are equivalent:
brain.find({ where: { status: 'active' } })
brain.find({ where: { status: { equals: 'active' } } })
```
---
## Comparison
| Operator | Alias | Description | Example |
|----------|-------|-------------|---------|
| `greaterThan` | `gt` | Greater than | `{ age: { greaterThan: 18 } }` |
| `greaterEqual` | `gte` | Greater or equal | `{ score: { greaterEqual: 90 } }` |
| `lessThan` | `lt` | Less than | `{ price: { lessThan: 100 } }` |
| `lessEqual` | `lte` | Less or equal | `{ rating: { lessEqual: 3 } }` |
| `between` | — | Inclusive range `[min, max]` | `{ year: { between: [2020, 2025] } }` |
```typescript
// Range query
const recent = await brain.find({
where: {
createdAt: { between: [Date.now() - 86400000, Date.now()] }
}
})
```
---
## Array / Set
| Operator | Alias | Description | Example |
|----------|-------|-------------|---------|
| `oneOf` | `in` | Value is one of the given options | `{ color: { oneOf: ['red', 'blue'] } }` |
| `noneOf` | — | Value is NOT one of the given options | `{ status: { noneOf: ['deleted', 'archived'] } }` |
| `contains` | — | Array field contains value | `{ tags: { contains: 'ai' } }` |
| `excludes` | — | Array field does NOT contain value | `{ tags: { excludes: 'spam' } }` |
| `hasAll` | — | Array field contains ALL listed values | `{ skills: { hasAll: ['js', 'ts'] } }` |
```typescript
// Find entities tagged with 'ai'
const aiEntities = await brain.find({
where: { tags: { contains: 'ai' } }
})
// Find entities of specific types
const people = await brain.find({
where: { noun: { oneOf: ['Person', 'Agent'] } }
})
```
---
## Existence
| Operator | Description | Example |
|----------|-------------|---------|
| `exists: true` | Field exists (has any value) | `{ email: { exists: true } }` |
| `exists: false` | Field does NOT exist | `{ email: { exists: false } }` |
| `missing: true` | Field does NOT exist (alias for `exists: false`) | `{ email: { missing: true } }` |
| `missing: false` | Field exists (alias for `exists: true`) | `{ email: { missing: false } }` |
```typescript
// Find entities that have an email field
const withEmail = await brain.find({
where: { email: { exists: true } }
})
```
---
## Pattern (In-Memory Only)
These operators work via the in-memory filter path. They are applied **after** the indexed query, so use them with other indexed operators for best performance.
| Operator | Description | Example |
|----------|-------------|---------|
| `matches` | Regex or string pattern match | `{ name: { matches: /^Dr\./ } }` |
| `startsWith` | String prefix | `{ name: { startsWith: 'John' } }` |
| `endsWith` | String suffix | `{ email: { endsWith: '@gmail.com' } }` |
```typescript
const doctors = await brain.find({
where: {
type: NounType.Person, // Indexed — fast
name: { startsWith: 'Dr.' } // In-memory — applied after
}
})
```
---
## Logical
Combine multiple conditions:
| Operator | Description | Example |
|----------|-------------|---------|
| `allOf` | ALL sub-filters must match (AND) | `{ allOf: [{ status: 'active' }, { role: 'admin' }] }` |
| `anyOf` | ANY sub-filter must match (OR) | `{ anyOf: [{ role: 'admin' }, { role: 'owner' }] }` |
| `not` | Invert a filter | `{ not: { status: 'deleted' } }` |
```typescript
// Complex OR query
const adminsOrOwners = await brain.find({
where: {
anyOf: [
{ role: 'admin' },
{ role: 'owner' }
]
}
})
// NOT query
const notDeleted = await brain.find({
where: {
not: { status: 'deleted' }
}
})
// Combined AND + OR
const results = await brain.find({
where: {
allOf: [
{ department: 'engineering' },
{ anyOf: [
{ level: 'senior' },
{ yearsExperience: { greaterThan: 5 } }
]}
]
}
})
```
---
## Indexed vs In-Memory Operators
Brainy's MetadataIndex supports a subset of operators natively for O(1) field lookups. Other operators fall back to in-memory filtering.
| Operator | MetadataIndex (Indexed) | In-Memory Fallback |
|----------|:-----------------------:|:------------------:|
| `equals` / `eq` | Yes | Yes |
| `notEquals` / `ne` | — | Yes |
| `greaterThan` / `gt` | Yes | Yes |
| `greaterEqual` / `gte` | Yes | Yes |
| `lessThan` / `lt` | Yes | Yes |
| `lessEqual` / `lte` | Yes | Yes |
| `between` | Yes | Yes |
| `oneOf` / `in` | Yes | Yes |
| `noneOf` | — | Yes |
| `contains` | Yes | Yes |
| `exists` / `missing` | Yes | Yes |
| `matches` | — | Yes |
| `startsWith` | — | Yes |
| `endsWith` | — | Yes |
| `allOf` | Partial | Yes |
| `anyOf` | Partial | Yes |
| `not` | — | Yes |
**Performance tip:** Combine indexed operators (equals, greaterThan, oneOf, between, contains, exists) with pattern operators for optimal speed — the index narrows results first, then patterns filter in memory.
---
## Practical Examples
### Filter by entity type
```typescript
// Using the type shorthand (recommended)
brain.find({ type: NounType.Person })
// Using where.noun directly
brain.find({ where: { noun: NounType.Person } })
// Multiple types
brain.find({ type: [NounType.Person, NounType.Agent] })
```
feat: subtype top-level field + trackField + migrateField Promotes `subtype?: string` to a top-level standard field on every entity, alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the consumer-chosen vocabulary for sub-classifying entities within a NounType (Person → employee/customer, Document → invoice/contract, etc.). Layer 1 — subtype field + rollup - HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry - Entity / Result / AddParams / UpdateParams / FindParams threading - add()/update() persist subtype on storageMetadata + entityForIndexing - get()/find() route through the standard-field fast path - subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on BaseStorage, mirrored after nounCountsByType with the same self-heal rebuild and persisted to _system/subtype-statistics.json - brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown - brain.counts.topSubtypes(type, n) — top-N by count - brain.subtypesOf(type) — distinct subtypes seen - find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path Layer 2 — trackField for other facets - brain.trackField(name, { perType?, values? }) registers a field for cardinality + per-NounType breakdown stats. Backed by the aggregation engine (auto-defines __fieldCounts__<name>), backfill-on-define applies. - brain.counts.byField(name, { type? }) returns value frequencies - Optional vocabulary whitelist rejects off-vocabulary writes at add/update Layer 3 — generic migrateField - brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? }) streams every entity, copies the value from one path to another, and (unless readBoth) clears the source. Supports top-level standard fields, metadata.X, and data.X paths. Idempotent — safe to re-run. Docs - New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3) - README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system, quick-start all treat subtype as a core primitive with anonymous example vocabularies (employee/customer/invoice/milestone). Tests - 26 new integration tests covering write/read/update/delete round-trips, counts rollup decrement + re-route on mutation, trackField + byField with and without perType, vocabulary whitelist enforcement, and migrateField for metadata.X → subtype and data.X → subtype paths including readBoth deprecation-window semantics. Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:24:36 -07:00
### Filter by subtype
`subtype` is a top-level standard field — takes the column-store fast path, not the metadata fallback. Pair with `type` for the typical "Person who is an employee" query:
```typescript
// Equality on subtype:
brain.find({ type: NounType.Person, subtype: 'employee' })
// Set membership:
brain.find({ type: NounType.Person, subtype: ['employee', 'contractor'] })
// Operator-form predicates use `where`:
brain.find({
type: NounType.Person,
where: { subtype: { exists: true } }
})
```
See the **[Subtypes & Facets guide](./guides/subtypes-and-facets.md)** for the full surface.
### Combine semantic search with filters
```typescript
const results = await brain.find({
query: 'machine learning engineer', // Semantic search (on data)
type: NounType.Person, // Type filter (indexed)
where: {
department: 'engineering', // Exact match (indexed)
yearsExperience: { greaterThan: 3 } // Range filter (indexed)
},
limit: 10
})
```
### Temporal queries
```typescript
const lastWeek = Date.now() - 7 * 24 * 60 * 60 * 1000
const recentEntities = await brain.find({
where: {
createdAt: { greaterThan: lastWeek }
},
orderBy: 'createdAt',
order: 'desc',
limit: 50
})
```
### Graph + metadata combination
```typescript
const results = await brain.find({
connected: {
from: teamLeadId,
via: VerbType.WorksWith,
depth: 2
},
where: {
role: { oneOf: ['engineer', 'designer'] },
active: true
}
})
```
---
## See Also
- [Data Model](./DATA_MODEL.md) — Entity structure, data vs metadata
- [API Reference](./api/README.md) — Complete API documentation
- [Find System](./FIND_SYSTEM.md) — Natural language find() details