brainy/docs/QUERY_OPERATORS.md
David Snelling 2cdf70ee0f feat: subtype top-level field + trackField + migrateField
Promotes `subtype?: string` to a top-level standard field on every entity,
alongside `type` / `confidence` / `weight`. Flat string, no hierarchy — the
consumer-chosen vocabulary for sub-classifying entities within a NounType
(Person → employee/customer, Document → invoice/contract, etc.).

Layer 1 — subtype field + rollup
- HNSWNounWithMetadata.subtype + STANDARD_ENTITY_FIELDS entry
- Entity / Result / AddParams / UpdateParams / FindParams threading
- add()/update() persist subtype on storageMetadata + entityForIndexing
- get()/find() route through the standard-field fast path
- subtypeCountsByType (Map<NounTypeIdx, Map<subtype, count>>) on
  BaseStorage, mirrored after nounCountsByType with the same self-heal
  rebuild and persisted to _system/subtype-statistics.json
- brain.counts.bySubtype(type, subtype?) — O(1) point + breakdown
- brain.counts.topSubtypes(type, n) — top-N by count
- brain.subtypesOf(type) — distinct subtypes seen
- find({ type, subtype }) and find({ subtype: ['a','b'] }) on the fast path

Layer 2 — trackField for other facets
- brain.trackField(name, { perType?, values? }) registers a field for
  cardinality + per-NounType breakdown stats. Backed by the aggregation
  engine (auto-defines __fieldCounts__<name>), backfill-on-define applies.
- brain.counts.byField(name, { type? }) returns value frequencies
- Optional vocabulary whitelist rejects off-vocabulary writes at add/update

Layer 3 — generic migrateField
- brain.migrateField({ from, to, readBoth?, batchSize?, onProgress? })
  streams every entity, copies the value from one path to another, and
  (unless readBoth) clears the source. Supports top-level standard fields,
  metadata.X, and data.X paths. Idempotent — safe to re-run.

Docs
- New guide: docs/guides/subtypes-and-facets.md (Layer 1 + 2 + 3)
- README, DATA_MODEL, QUERY_OPERATORS, api/README, finite-type-system,
  quick-start all treat subtype as a core primitive with anonymous example
  vocabularies (employee/customer/invoice/milestone).

Tests
- 26 new integration tests covering write/read/update/delete round-trips,
  counts rollup decrement + re-route on mutation, trackField + byField
  with and without perType, vocabulary whitelist enforcement, and
  migrateField for metadata.X → subtype and data.X → subtype paths
  including readBoth deprecation-window semantics.

Unit suite: 1468/1468 passing. Type-check + build clean.
2026-06-04 17:25:28 -07:00

7.3 KiB

Query Operators (BFO)

Brainy Field Operators — the complete reference for where filters in find().

All operators work with find({ where: { ... } }) and filter on metadata fields (not data).


Equality

Operator Alias Description Example
equals eq, is Exact match { status: { equals: 'active' } }
notEquals ne, isNot Not equal { status: { notEquals: 'deleted' } }

Shorthand: A bare value is treated as equals:

// These are equivalent:
brain.find({ where: { status: 'active' } })
brain.find({ where: { status: { equals: 'active' } } })

Comparison

Operator Alias Description Example
greaterThan gt Greater than { age: { greaterThan: 18 } }
greaterEqual gte Greater or equal { score: { greaterEqual: 90 } }
lessThan lt Less than { price: { lessThan: 100 } }
lessEqual lte Less or equal { rating: { lessEqual: 3 } }
between Inclusive range [min, max] { year: { between: [2020, 2025] } }
// Range query
const recent = await brain.find({
  where: {
    createdAt: { between: [Date.now() - 86400000, Date.now()] }
  }
})

Array / Set

Operator Alias Description Example
oneOf in Value is one of the given options { color: { oneOf: ['red', 'blue'] } }
noneOf Value is NOT one of the given options { status: { noneOf: ['deleted', 'archived'] } }
contains Array field contains value { tags: { contains: 'ai' } }
excludes Array field does NOT contain value { tags: { excludes: 'spam' } }
hasAll Array field contains ALL listed values { skills: { hasAll: ['js', 'ts'] } }
// Find entities tagged with 'ai'
const aiEntities = await brain.find({
  where: { tags: { contains: 'ai' } }
})

// Find entities of specific types
const people = await brain.find({
  where: { noun: { oneOf: ['Person', 'Agent'] } }
})

Existence

Operator Description Example
exists: true Field exists (has any value) { email: { exists: true } }
exists: false Field does NOT exist { email: { exists: false } }
missing: true Field does NOT exist (alias for exists: false) { email: { missing: true } }
missing: false Field exists (alias for exists: true) { email: { missing: false } }
// Find entities that have an email field
const withEmail = await brain.find({
  where: { email: { exists: true } }
})

Pattern (In-Memory Only)

These operators work via the in-memory filter path. They are applied after the indexed query, so use them with other indexed operators for best performance.

Operator Description Example
matches Regex or string pattern match { name: { matches: /^Dr\./ } }
startsWith String prefix { name: { startsWith: 'John' } }
endsWith String suffix { email: { endsWith: '@gmail.com' } }
const doctors = await brain.find({
  where: {
    type: NounType.Person,           // Indexed — fast
    name: { startsWith: 'Dr.' }     // In-memory — applied after
  }
})

Logical

Combine multiple conditions:

Operator Description Example
allOf ALL sub-filters must match (AND) { allOf: [{ status: 'active' }, { role: 'admin' }] }
anyOf ANY sub-filter must match (OR) { anyOf: [{ role: 'admin' }, { role: 'owner' }] }
not Invert a filter { not: { status: 'deleted' } }
// Complex OR query
const adminsOrOwners = await brain.find({
  where: {
    anyOf: [
      { role: 'admin' },
      { role: 'owner' }
    ]
  }
})

// NOT query
const notDeleted = await brain.find({
  where: {
    not: { status: 'deleted' }
  }
})

// Combined AND + OR
const results = await brain.find({
  where: {
    allOf: [
      { department: 'engineering' },
      { anyOf: [
        { level: 'senior' },
        { yearsExperience: { greaterThan: 5 } }
      ]}
    ]
  }
})

Indexed vs In-Memory Operators

Brainy's MetadataIndex supports a subset of operators natively for O(1) field lookups. Other operators fall back to in-memory filtering.

Operator MetadataIndex (Indexed) In-Memory Fallback
equals / eq Yes Yes
notEquals / ne Yes
greaterThan / gt Yes Yes
greaterEqual / gte Yes Yes
lessThan / lt Yes Yes
lessEqual / lte Yes Yes
between Yes Yes
oneOf / in Yes Yes
noneOf Yes
contains Yes Yes
exists / missing Yes Yes
matches Yes
startsWith Yes
endsWith Yes
allOf Partial Yes
anyOf Partial Yes
not Yes

Performance tip: Combine indexed operators (equals, greaterThan, oneOf, between, contains, exists) with pattern operators for optimal speed — the index narrows results first, then patterns filter in memory.


Practical Examples

Filter by entity type

// Using the type shorthand (recommended)
brain.find({ type: NounType.Person })

// Using where.noun directly
brain.find({ where: { noun: NounType.Person } })

// Multiple types
brain.find({ type: [NounType.Person, NounType.Agent] })

Filter by subtype

subtype is a top-level standard field — takes the column-store fast path, not the metadata fallback. Pair with type for the typical "Person who is an employee" query:

// Equality on subtype:
brain.find({ type: NounType.Person, subtype: 'employee' })

// Set membership:
brain.find({ type: NounType.Person, subtype: ['employee', 'contractor'] })

// Operator-form predicates use `where`:
brain.find({
  type: NounType.Person,
  where: { subtype: { exists: true } }
})

See the Subtypes & Facets guide for the full surface.

Combine semantic search with filters

const results = await brain.find({
  query: 'machine learning engineer',     // Semantic search (on data)
  type: NounType.Person,                  // Type filter (indexed)
  where: {
    department: 'engineering',            // Exact match (indexed)
    yearsExperience: { greaterThan: 3 }   // Range filter (indexed)
  },
  limit: 10
})

Temporal queries

const lastWeek = Date.now() - 7 * 24 * 60 * 60 * 1000
const recentEntities = await brain.find({
  where: {
    createdAt: { greaterThan: lastWeek }
  },
  orderBy: 'createdAt',
  order: 'desc',
  limit: 50
})

Graph + metadata combination

const results = await brain.find({
  connected: {
    from: teamLeadId,
    via: VerbType.WorksWith,
    depth: 2
  },
  where: {
    role: { oneOf: ['engineer', 'designer'] },
    active: true
  }
})

See Also