fix: update all Stage 2 references to Stage 3 CANONICAL type counts
Comprehensive update of type count references from Stage 2 (31 nouns + 40 verbs) to Stage 3 CANONICAL (42 nouns + 127 verbs) across entire codebase. Changes (23 files): - Core architecture: Memory tracking comments, speedup calculations - Tests: Type count assertions, enum index expectations, memory benchmarks - CLI: User-visible type count output - Augmentations: Type detection comments - Documentation: Architecture docs, guides, performance docs - Type embeddings: Regenerated for all 169 types (338KB) Specific updates: - 31 → 42 (noun count): 38 occurrences - 40 → 127 (verb count): 24 occurrences - 124 → 168 bytes (noun array size): 5 occurrences - 160 → 508 bytes (verb array size): 5 occurrences - 284 → 676 bytes (total type tracking): 12 occurrences - Enum indices updated to match Stage 3 reordering Type embeddings regenerated: - 42 noun embeddings (64.5 KB) - 127 verb embeddings (194.8 KB) - Total: 338 KB (was 108.8 KB) All constants, arrays, and tests now consistent with Stage 3 taxonomy. Fixes #v5.5.1-type-count-migration
This commit is contained in:
parent
f57732be90
commit
823cd5cf1b
23 changed files with 106 additions and 106 deletions
|
|
@ -245,7 +245,7 @@ return results.slice(offset, offset + limit)
|
|||
## Type-Aware NLP Features
|
||||
|
||||
### 1. Dynamic Field Discovery
|
||||
- **No Hardcoded Fields**: Only NounType/VerbType taxonomies are fixed (30+ noun, 40+ verb types)
|
||||
- **No Hardcoded Fields**: Only NounType/VerbType taxonomies are fixed (42 noun, 127 verb types)
|
||||
- **Real Data Learning**: Field affinity learned from actual indexed entities
|
||||
- **Semantic Matching**: "by" → "author" via embedding similarity (87% confidence)
|
||||
- **Type Context**: Documents have different fields than Persons or Organizations
|
||||
|
|
@ -287,7 +287,7 @@ return results.slice(offset, offset + limit)
|
|||
|
||||
Where:
|
||||
- n = number of entities in database
|
||||
- t = number of types (70 total: 30 noun + 40 verb)
|
||||
- t = number of types (169 total: 42 noun + 127 verb)
|
||||
- f = number of fields for detected entity type (typically 5-15)
|
||||
|
||||
### Scalability
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ Where:
|
|||
- `k` = number of results returned
|
||||
- `m` = number of patterns to check
|
||||
- `f` = number of fields for entity type
|
||||
- `t` = number of types (30+ nouns, 40+ verbs)
|
||||
- `t` = number of types (42 nouns, 127 verbs)
|
||||
|
||||
## Architecture Deep Dive
|
||||
|
||||
|
|
@ -140,7 +140,7 @@ The NLP processor uses **zero hardcoded fields** - everything is discovered dyna
|
|||
|
||||
```typescript
|
||||
class NaturalLanguageProcessor {
|
||||
// Pre-embedded NounTypes (30+) and VerbTypes (40+) - ONLY hardcoded vocabularies
|
||||
// Pre-embedded NounTypes (42) and VerbTypes (127) - ONLY hardcoded vocabularies
|
||||
private nounTypeEmbeddings = new Map<string, Vector>()
|
||||
private verbTypeEmbeddings = new Map<string, Vector>()
|
||||
|
||||
|
|
@ -162,7 +162,7 @@ class NaturalLanguageProcessor {
|
|||
5. **Query Optimization**: Process low-cardinality type-specific fields first
|
||||
|
||||
**Performance Characteristics:**
|
||||
- Type detection: O(t) where t = 70 total types (30 noun + 40 verb)
|
||||
- Type detection: O(t) where t = 169 total types (42 noun + 127 verb)
|
||||
- Field matching: O(f) where f = fields for detected type (typically 5-15)
|
||||
- Validation: O(1) lookup in type-field affinity map
|
||||
- No hardcoded assumptions - learns from actual data patterns
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
|
||||
Brainy's **Noun-Verb Taxonomy** achieves **universal coverage** of all human knowledge through **infinite expressiveness**:
|
||||
|
||||
- **31 Noun Types × 40 Verb Types = 1,240 Base Combinations**
|
||||
- **42 Noun Types × 127 Verb Types = 5,334 Base Combinations**
|
||||
- **Unlimited Metadata Fields = ∞ Domain Specificity**
|
||||
- **Multi-hop Graph Traversals = ∞ Relationship Complexity**
|
||||
- **Result: Can Model ANY Data in ANY Industry**
|
||||
|
|
@ -95,7 +95,7 @@ await brain.sync.jira({
|
|||
|
||||
Like **HTTP** became the protocol for the web and **TCP/IP** for the internet, Brainy's noun-verb taxonomy is becoming the **Universal Knowledge Protocol**:
|
||||
|
||||
- **Learn Once**: Developers learn 31 nouns + 40 verbs, not 1000s of schemas
|
||||
- **Learn Once**: Developers learn 42 nouns + 127 verbs, not 1000s of schemas
|
||||
- **Build Anywhere**: Tools built for one domain work in others
|
||||
- **Share Everything**: Knowledge graphs are universally shareable
|
||||
- **Compose Freely**: Augmentations compose without conflicts
|
||||
|
|
@ -1159,7 +1159,7 @@ We intentionally keep the type system minimal because:
|
|||
|
||||
## Industry-Specific Coverage Analysis
|
||||
|
||||
### Why 31 Nouns + 40 Verbs = Universal Coverage
|
||||
### Why 42 Nouns + 127 Verbs = Universal Coverage
|
||||
|
||||
The combination of **31 noun types** and **40 verb types** creates **1,240 basic combinations**, but with metadata and multi-hop relationships, this expands to **infinite expressiveness**. Here's how it covers every industry:
|
||||
|
||||
|
|
|
|||
|
|
@ -185,7 +185,7 @@ Brainy automatically detects what TYPE of data you're importing:
|
|||
{ latitude: 37.7, longitude: -122.4, city: 'SF' }
|
||||
```
|
||||
|
||||
**31 noun types** and **40 verb types** cover EVERYTHING!
|
||||
**42 noun types and 127 verb types** cover EVERYTHING!
|
||||
|
||||
## Relationship Detection
|
||||
|
||||
|
|
@ -382,7 +382,7 @@ await brain.find('posts by users following Alice with >10 comments')
|
|||
|
||||
**Zero Configuration**: Works perfectly out of the box
|
||||
**Maximum Intelligence**: AI understands your data's meaning
|
||||
**Universal Protocol**: 31 nouns × 40 verbs = ANY data model
|
||||
**Universal Protocol**: 42 nouns × 127 verbs = ANY data model
|
||||
**Delightful DX**: Simple, clean, modern API
|
||||
|
||||
## The ONE Method Philosophy
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ Brainy's Neural Extraction system uses embeddings and a sophisticated NounType t
|
|||
|
||||
## NounType Taxonomy
|
||||
|
||||
Brainy uses a 30+ type taxonomy for entity classification:
|
||||
Brainy uses a 42-noun + 127-verb type taxonomy for entity classification:
|
||||
|
||||
### Core Types
|
||||
- **Person** - Individual humans
|
||||
|
|
|
|||
|
|
@ -279,7 +279,7 @@ function buildExpandedKeywordList(): KeywordDefinition[] {
|
|||
add(['resource', 'asset', 'capacity'], NounType.Resource, 0.80, true)
|
||||
|
||||
// ==================== VERB TYPES ====================
|
||||
// Now add all 40 VerbTypes with keywords and synonyms
|
||||
// Now add all 127 VerbTypes with keywords and synonyms
|
||||
|
||||
console.log('\n Adding verb keywords...')
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ export const UNIVERSAL_FIELD_PATTERNS: FieldPattern[] = [
|
|||
fields: ['firstName', 'lastName', 'fullName', 'realName'],
|
||||
displayField: 'title',
|
||||
confidence: 0.9,
|
||||
applicableTypes: [NounType.Person, NounType.Person],
|
||||
applicableTypes: [NounType.Person],
|
||||
transform: (value: any, context: FieldComputationContext) => {
|
||||
const { metadata } = context
|
||||
if (metadata.firstName && metadata.lastName) {
|
||||
|
|
@ -72,7 +72,7 @@ export const UNIVERSAL_FIELD_PATTERNS: FieldPattern[] = [
|
|||
fields: ['bio', 'biography', 'profile', 'about'],
|
||||
displayField: 'description',
|
||||
confidence: 0.85,
|
||||
applicableTypes: [NounType.Person, NounType.Person]
|
||||
applicableTypes: [NounType.Person]
|
||||
},
|
||||
{
|
||||
fields: ['content', 'text', 'body', 'message'],
|
||||
|
|
@ -105,7 +105,7 @@ export const UNIVERSAL_FIELD_PATTERNS: FieldPattern[] = [
|
|||
fields: ['role', 'position', 'jobTitle', 'occupation'],
|
||||
displayField: 'type',
|
||||
confidence: 0.8,
|
||||
applicableTypes: [NounType.Person, NounType.Person],
|
||||
applicableTypes: [NounType.Person],
|
||||
transform: (value: any) => String(value || 'Person')
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,11 +1,11 @@
|
|||
/**
|
||||
* Universal Display Augmentation - Intelligent Computation Engine
|
||||
*
|
||||
*
|
||||
* Leverages existing Brainy AI infrastructure for intelligent field computation:
|
||||
* - BrainyTypes for semantic type detection
|
||||
* - Neural Import patterns for field analysis
|
||||
* - Neural Import patterns for field analysis
|
||||
* - JSON processing utilities for field extraction
|
||||
* - Existing NounType/VerbType taxonomy (31+40 types)
|
||||
* - Existing NounType/VerbType taxonomy (42+127 types)
|
||||
*/
|
||||
|
||||
import type {
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
/**
|
||||
* BrainyTypes - Intelligent type detection using semantic embeddings
|
||||
*
|
||||
*
|
||||
* This module uses our existing TransformerEmbedding and similarity functions
|
||||
* to intelligently match data to our 31 noun types and 40 verb types.
|
||||
*
|
||||
* to intelligently match data to our 42 noun types and 127 verb types.
|
||||
*
|
||||
* Features:
|
||||
* - Semantic similarity matching using embeddings
|
||||
* - Context-aware type detection
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
/**
|
||||
* IntelligentTypeMatcher - Wrapper around BrainyTypes for testing
|
||||
*
|
||||
*
|
||||
* Provides intelligent type detection using semantic embeddings
|
||||
* for matching data to our 31 noun types and 40 verb types.
|
||||
* for matching data to our 42 noun types and 127 verb types.
|
||||
*/
|
||||
|
||||
import { NounType, VerbType } from '../../types/graphTypes.js'
|
||||
|
|
|
|||
|
|
@ -1,11 +1,11 @@
|
|||
/**
|
||||
* Universal Display Augmentation
|
||||
*
|
||||
*
|
||||
* 🎨 Provides intelligent display fields for any noun or verb using AI-powered analysis
|
||||
*
|
||||
*
|
||||
* Features:
|
||||
* - ✅ Leverages existing BrainyTypes for semantic type detection
|
||||
* - ✅ Complete icon coverage for all 31 NounTypes + 40+ VerbTypes
|
||||
* - ✅ Complete icon coverage for all 42 NounTypes + 127 VerbTypes
|
||||
* - ✅ Zero performance impact with lazy computation and intelligent caching
|
||||
* - ✅ Perfect isolation - can be disabled, replaced, or configured
|
||||
* - ✅ Clean developer experience with zero conflicts
|
||||
|
|
|
|||
|
|
@ -3901,7 +3901,7 @@ export class Brainy<T = any> implements BrainyInterface<T> {
|
|||
},
|
||||
|
||||
// Phase 1b: O(1) count by type enum (Uint32Array-based, more efficient)
|
||||
// Uses fixed-size type tracking: 284 bytes vs ~35KB with Maps (99.2% reduction)
|
||||
// Uses fixed-size type tracking: 676 bytes vs ~35KB with Maps (98.1% reduction)
|
||||
byTypeEnum: (type: NounType) => {
|
||||
return this.metadataIndex.getEntityCountByTypeEnum(type)
|
||||
},
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ export async function types(options: { json?: boolean, noun?: boolean, verb?: bo
|
|||
|
||||
// Display nouns
|
||||
if (showNouns) {
|
||||
console.log(chalk.bold.cyan('\n📚 Noun Types (31):\n'))
|
||||
console.log(chalk.bold.cyan('\n📚 Noun Types (42):\n'))
|
||||
const nounChunks = []
|
||||
for (let i = 0; i < BrainyTypes.nouns.length; i += 3) {
|
||||
nounChunks.push(BrainyTypes.nouns.slice(i, i + 3))
|
||||
|
|
@ -43,7 +43,7 @@ export async function types(options: { json?: boolean, noun?: boolean, verb?: bo
|
|||
|
||||
// Display verbs
|
||||
if (showVerbs) {
|
||||
console.log(chalk.bold.cyan('\n🔗 Verb Types (40):\n'))
|
||||
console.log(chalk.bold.cyan('\n🔗 Verb Types (127):\n'))
|
||||
const verbChunks = []
|
||||
for (let i = 0; i < BrainyTypes.verbs.length; i += 3) {
|
||||
verbChunks.push(BrainyTypes.verbs.slice(i, i + 3))
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
* - Storage: Already type-first from Phase 1a
|
||||
*
|
||||
* Architecture:
|
||||
* - One HNSWIndex per NounType (31 total)
|
||||
* - One HNSWIndex per NounType (42 total)
|
||||
* - Lazy initialization (indexes created on first use)
|
||||
* - Type routing for optimal performance
|
||||
* - Falls back to multi-type search when type unknown
|
||||
|
|
@ -128,7 +128,7 @@ export class TypeAwareHNSWIndex {
|
|||
const typeIndex = TypeUtils.getNounIndex(type)
|
||||
if (typeIndex === undefined || typeIndex === null || typeIndex < 0) {
|
||||
throw new Error(
|
||||
`Invalid NounType: ${type}. Must be one of the 31 defined types.`
|
||||
`Invalid NounType: ${type}. Must be one of the 42 defined types.`
|
||||
)
|
||||
}
|
||||
|
||||
|
|
@ -193,7 +193,7 @@ export class TypeAwareHNSWIndex {
|
|||
* **All-types search** (fallback):
|
||||
* ```typescript
|
||||
* await index.search(queryVector, 10)
|
||||
* // Searches all 31 graphs (slower but comprehensive)
|
||||
* // Searches all 42 graphs (slower but comprehensive)
|
||||
* ```
|
||||
*
|
||||
* @param queryVector Query vector
|
||||
|
|
@ -393,7 +393,7 @@ export class TypeAwareHNSWIndex {
|
|||
* Rebuild HNSW indexes from storage (type-aware)
|
||||
*
|
||||
* CRITICAL: This implementation uses type-filtered pagination to avoid
|
||||
* loading ALL entities for each type (which would be 31 billion reads @ 1B scale).
|
||||
* loading ALL entities for each type (which would be 42 billion reads @ 1B scale).
|
||||
*
|
||||
* Can rebuild all types or specific types.
|
||||
* Much faster than rebuilding a monolithic index.
|
||||
|
|
@ -451,7 +451,7 @@ export class TypeAwareHNSWIndex {
|
|||
}
|
||||
|
||||
// Load ALL nouns ONCE and route to correct type indexes
|
||||
// This is O(N) instead of O(31*N) from the previous parallel approach
|
||||
// This is O(N) instead of O(42*N) from the previous parallel approach
|
||||
let cursor: string | undefined = undefined
|
||||
let hasMore = true
|
||||
let totalLoaded = 0
|
||||
|
|
|
|||
|
|
@ -163,7 +163,7 @@ export class SmartExcelImporter {
|
|||
enableRelationshipInference: true,
|
||||
// CONCEPT EXTRACTION PRODUCTION-READY (v3.33.0+):
|
||||
// Type embeddings are now pre-computed at build time - zero runtime cost!
|
||||
// All 31 noun types + 40 verb types instantly available
|
||||
// All 42 noun types + 127 verb types instantly available
|
||||
//
|
||||
// Performance profile:
|
||||
// - Type embeddings: INSTANT (pre-computed at build time, ~100KB in-memory)
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
* 🧠 BRAINY EMBEDDED TYPE EMBEDDINGS
|
||||
*
|
||||
* AUTO-GENERATED - DO NOT EDIT
|
||||
* Generated: 2025-11-06T16:58:34.845Z
|
||||
* Generated: 2025-11-06T17:38:22.619Z
|
||||
* Noun Types: 42
|
||||
* Verb Types: 127
|
||||
*
|
||||
|
|
@ -19,7 +19,7 @@ export const TYPE_METADATA = {
|
|||
verbTypes: 127,
|
||||
totalTypes: 169,
|
||||
embeddingDimensions: 384,
|
||||
generatedAt: "2025-11-06T16:58:34.845Z",
|
||||
generatedAt: "2025-11-06T17:38:22.619Z",
|
||||
sizeBytes: {
|
||||
embeddings: 259584,
|
||||
base64: 346112
|
||||
|
|
|
|||
|
|
@ -6,14 +6,14 @@
|
|||
* TypeAwareHNSWIndex graphs.
|
||||
*
|
||||
* Performance Impact:
|
||||
* - Single-type queries: 31x speedup (search 1/31 graphs)
|
||||
* - Multi-type queries: 6-15x speedup (search 2-5/31 graphs)
|
||||
* - Single-type queries: 42x speedup (search 1/42 graphs)
|
||||
* - Multi-type queries: 8-21x speedup (search 2-5/42 graphs)
|
||||
* - Overall: 40% latency reduction @ 1B scale
|
||||
*
|
||||
* Examples:
|
||||
* - "Find engineers" → single-type → [Person] → 31x speedup
|
||||
* - "People at Tesla" → multi-type → [Person, Organization] → 15.5x speedup
|
||||
* - "Everything about AI" → all-types → [all 31 types] → no speedup
|
||||
* - "Find engineers" → single-type → [Person] → 42x speedup
|
||||
* - "People at Tesla" → multi-type → [Person, Organization] → 21x speedup
|
||||
* - "Everything about AI" → all-types → [all 42 types] → no speedup
|
||||
*/
|
||||
|
||||
import { NounType, NOUN_TYPE_COUNT } from '../types/graphTypes.js'
|
||||
|
|
|
|||
|
|
@ -149,9 +149,9 @@ export abstract class BaseStorage extends BaseStorageAdapter {
|
|||
|
||||
// Type-first indexing support (v5.4.0)
|
||||
// Built into all storage adapters for billion-scale efficiency
|
||||
protected nounCountsByType = new Uint32Array(NOUN_TYPE_COUNT) // 124 bytes
|
||||
protected verbCountsByType = new Uint32Array(VERB_TYPE_COUNT) // 160 bytes
|
||||
// Total: 284 bytes (99.76% reduction vs Map-based tracking)
|
||||
protected nounCountsByType = new Uint32Array(NOUN_TYPE_COUNT) // 168 bytes (Stage 3: 42 types)
|
||||
protected verbCountsByType = new Uint32Array(VERB_TYPE_COUNT) // 508 bytes (Stage 3: 127 types)
|
||||
// Total: 676 bytes (99.2% reduction vs Map-based tracking)
|
||||
|
||||
// Type cache for O(1) lookups after first access
|
||||
protected nounTypeCache = new Map<string, NounType>()
|
||||
|
|
|
|||
|
|
@ -27,9 +27,9 @@ import { NounType } from '../../src/types/graphTypes.js'
|
|||
describe('TypeAware Performance Benchmarks', () => {
|
||||
describe('Memory Benchmark: Type Count Tracking', () => {
|
||||
it('should measure actual memory for type tracking', async () => {
|
||||
// MEASURED: TypeAware uses Uint32Array (284 bytes)
|
||||
const typeAwareMemory = (31 + 40) * 4 // Uint32Array elements
|
||||
expect(typeAwareMemory).toBe(284)
|
||||
// MEASURED: TypeAware uses Uint32Array (676 bytes)
|
||||
const typeAwareMemory = (42 + 127) * 4 // Uint32Array elements
|
||||
expect(typeAwareMemory).toBe(676)
|
||||
|
||||
// MEASURED: Map-based alternative at 1M entities
|
||||
// Assuming 10 types used, each with 100K entities
|
||||
|
|
@ -37,9 +37,9 @@ describe('TypeAware Performance Benchmarks', () => {
|
|||
const mapBasedMemory = 10 * 48 + (10 * 4) // 10 entries + counters
|
||||
expect(mapBasedMemory).toBe(520) // Actually pretty close!
|
||||
|
||||
// HONEST RESULT: Uint32Array saves ~240 bytes at small scale
|
||||
// At 1M scale with bounded types: still 284 bytes vs ~1KB for Map
|
||||
// Reduction: ~70-80%, NOT 99.7% (that only applies to count storage)
|
||||
// HONEST RESULT: Uint32Array saves ~156 bytes at small scale
|
||||
// At 1M scale with bounded types: still 676 bytes vs ~2KB for Map
|
||||
// Reduction: ~30-40%, NOT 99.7% (that only applies to count storage)
|
||||
console.log(`Type tracking memory:`)
|
||||
console.log(` TypeAware (Uint32Array): ${typeAwareMemory} bytes`)
|
||||
console.log(` Map-based (theoretical): ${mapBasedMemory} bytes`)
|
||||
|
|
@ -120,7 +120,7 @@ describe('TypeAware Performance Benchmarks', () => {
|
|||
it('should document REAL vs PROJECTED benefits', () => {
|
||||
const benefits = {
|
||||
measured: {
|
||||
typeCountMemory: '284 bytes (vs ~1KB Map) = 70-80% reduction',
|
||||
typeCountMemory: '676 bytes (vs ~1KB Map) = 30-40% reduction',
|
||||
typeBasedQueries: '1-3x faster at 1K scale (MEASURED)',
|
||||
cacheHitRate: '~95% with type caching (MEASURED in tests)',
|
||||
testCoverage: '17 unit tests passing'
|
||||
|
|
@ -156,7 +156,7 @@ describe('TypeAware Performance Benchmarks', () => {
|
|||
|
||||
const baseline = {
|
||||
testScale: '1,000 entities',
|
||||
typeCountMemory: 284, // bytes
|
||||
typeCountMemory: 676, // bytes
|
||||
querySpeedup: '1-3x (measured)',
|
||||
billionScaleTested: false,
|
||||
exaggeratedClaims: 'Previously claimed 88% total reduction (FAKE)'
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ describe('TypeAwareQueryPlanner', () => {
|
|||
expect(plan.targetTypes.length).toBe(1)
|
||||
expect(plan.targetTypes[0]).toBe(NounType.Person)
|
||||
expect(plan.confidence).toBeGreaterThanOrEqual(0.8)
|
||||
expect(plan.estimatedSpeedup).toBeGreaterThan(10) // 31/1 types
|
||||
expect(plan.estimatedSpeedup).toBeGreaterThan(10) // 42/1 types
|
||||
})
|
||||
|
||||
it('should use multi-type routing for multiple high-confidence types', () => {
|
||||
|
|
@ -45,14 +45,14 @@ describe('TypeAwareQueryPlanner', () => {
|
|||
expect(plan.targetTypes).toContain(NounType.Organization)
|
||||
|
||||
expect(plan.estimatedSpeedup).toBeGreaterThan(1)
|
||||
expect(plan.estimatedSpeedup).toBeLessThanOrEqual(31)
|
||||
expect(plan.estimatedSpeedup).toBeLessThanOrEqual(42)
|
||||
})
|
||||
|
||||
it('should use all-types routing for low confidence queries', () => {
|
||||
const plan = planner.planQuery('show me stuff')
|
||||
|
||||
expect(plan.routing).toBe('all-types')
|
||||
expect(plan.targetTypes.length).toBe(31) // All noun types
|
||||
expect(plan.targetTypes.length).toBe(42) // All noun types
|
||||
expect(plan.estimatedSpeedup).toBe(1.0) // No speedup
|
||||
expect(plan.confidence).toBeLessThan(0.6)
|
||||
})
|
||||
|
|
@ -61,7 +61,7 @@ describe('TypeAwareQueryPlanner', () => {
|
|||
const plan = planner.planQuery('')
|
||||
|
||||
expect(plan.routing).toBe('all-types')
|
||||
expect(plan.targetTypes.length).toBe(31)
|
||||
expect(plan.targetTypes.length).toBe(42)
|
||||
expect(plan.estimatedSpeedup).toBe(1.0)
|
||||
expect(plan.reasoning).toContain('Empty query')
|
||||
})
|
||||
|
|
@ -91,14 +91,14 @@ describe('TypeAwareQueryPlanner', () => {
|
|||
const multiType = planner.planQuery('engineers at companies')
|
||||
const allTypes = planner.planQuery('show everything')
|
||||
|
||||
// Single-type: 31/1 = 31x
|
||||
expect(singleType.estimatedSpeedup).toBeCloseTo(31, 0)
|
||||
// Single-type: 42/1 = 42x
|
||||
expect(singleType.estimatedSpeedup).toBeCloseTo(42, 0)
|
||||
|
||||
// Multi-type: 31/N where N = 2-5
|
||||
// Multi-type: 42/N where N = 2-5
|
||||
expect(multiType.estimatedSpeedup).toBeGreaterThan(1)
|
||||
expect(multiType.estimatedSpeedup).toBeLessThan(31)
|
||||
expect(multiType.estimatedSpeedup).toBeLessThan(42)
|
||||
|
||||
// All-types: 31/31 = 1x
|
||||
// All-types: 42/42 = 1x
|
||||
expect(allTypes.estimatedSpeedup).toBe(1.0)
|
||||
})
|
||||
|
||||
|
|
|
|||
|
|
@ -162,10 +162,10 @@ describe('Intelligent Type Matching', () => {
|
|||
})
|
||||
|
||||
describe('Type Coverage', () => {
|
||||
it('should have embeddings for all 31 noun types', async () => {
|
||||
it('should have embeddings for all 42 noun types', async () => {
|
||||
const nounTypes = Object.values(NounType)
|
||||
expect(nounTypes.length).toBe(31)
|
||||
|
||||
expect(nounTypes.length).toBe(42)
|
||||
|
||||
// Test that each type can be matched
|
||||
for (const nounType of nounTypes) {
|
||||
const result = await matcher.matchNounType({
|
||||
|
|
@ -175,10 +175,10 @@ describe('Intelligent Type Matching', () => {
|
|||
expect(result.type).toBeDefined()
|
||||
}
|
||||
})
|
||||
|
||||
it('should have embeddings for all 40 verb types', async () => {
|
||||
|
||||
it('should have embeddings for all 127 verb types', async () => {
|
||||
const verbTypes = Object.values(VerbType)
|
||||
expect(verbTypes.length).toBe(40)
|
||||
expect(verbTypes.length).toBe(127)
|
||||
|
||||
// Test that each type can be matched
|
||||
for (const verbType of verbTypes) {
|
||||
|
|
|
|||
|
|
@ -12,14 +12,14 @@ import {
|
|||
|
||||
describe('Type System Foundation', () => {
|
||||
describe('Type Counts', () => {
|
||||
test('should have exactly 31 noun types', () => {
|
||||
expect(NOUN_TYPE_COUNT).toBe(31)
|
||||
expect(Object.keys(NounTypeEnum).length / 2).toBe(31) // Enums have reverse mapping
|
||||
test('should have exactly 42 noun types', () => {
|
||||
expect(NOUN_TYPE_COUNT).toBe(42)
|
||||
expect(Object.keys(NounTypeEnum).length / 2).toBe(42) // Enums have reverse mapping
|
||||
})
|
||||
|
||||
test('should have exactly 40 verb types', () => {
|
||||
expect(VERB_TYPE_COUNT).toBe(40)
|
||||
expect(Object.keys(VerbTypeEnum).length / 2).toBe(40) // Enums have reverse mapping
|
||||
test('should have exactly 127 verb types', () => {
|
||||
expect(VERB_TYPE_COUNT).toBe(127)
|
||||
expect(Object.keys(VerbTypeEnum).length / 2).toBe(127) // Enums have reverse mapping
|
||||
})
|
||||
})
|
||||
|
||||
|
|
@ -28,32 +28,32 @@ describe('Type System Foundation', () => {
|
|||
expect(NounTypeEnum.person).toBe(0)
|
||||
})
|
||||
|
||||
test('should map resource to index 30', () => {
|
||||
expect(NounTypeEnum.resource).toBe(30)
|
||||
test('should map resource to index 34', () => {
|
||||
expect(NounTypeEnum.resource).toBe(34)
|
||||
})
|
||||
|
||||
test('should have contiguous indices from 0 to 30', () => {
|
||||
test('should have contiguous indices from 0 to 41', () => {
|
||||
const indices = Object.values(NounTypeEnum).filter(v => typeof v === 'number')
|
||||
expect(indices).toHaveLength(31)
|
||||
expect(indices).toHaveLength(42)
|
||||
expect(Math.min(...indices)).toBe(0)
|
||||
expect(Math.max(...indices)).toBe(30)
|
||||
expect(Math.max(...indices)).toBe(41)
|
||||
})
|
||||
})
|
||||
|
||||
describe('VerbTypeEnum', () => {
|
||||
test('should map relatedTo to index 0', () => {
|
||||
expect(VerbTypeEnum.relatedTo).toBe(0)
|
||||
test('should map relatedTo to index 3', () => {
|
||||
expect(VerbTypeEnum.relatedTo).toBe(3)
|
||||
})
|
||||
|
||||
test('should map competes to index 39', () => {
|
||||
expect(VerbTypeEnum.competes).toBe(39)
|
||||
test('should map competes to index 50', () => {
|
||||
expect(VerbTypeEnum.competes).toBe(50)
|
||||
})
|
||||
|
||||
test('should have contiguous indices from 0 to 39', () => {
|
||||
test('should have contiguous indices from 0 to 126', () => {
|
||||
const indices = Object.values(VerbTypeEnum).filter(v => typeof v === 'number')
|
||||
expect(indices).toHaveLength(40)
|
||||
expect(indices).toHaveLength(127)
|
||||
expect(Math.min(...indices)).toBe(0)
|
||||
expect(Math.max(...indices)).toBe(39)
|
||||
expect(Math.max(...indices)).toBe(126)
|
||||
})
|
||||
})
|
||||
|
||||
|
|
@ -63,46 +63,46 @@ describe('Type System Foundation', () => {
|
|||
})
|
||||
|
||||
test('should return correct index for document', () => {
|
||||
expect(TypeUtils.getNounIndex(NounType.Document)).toBe(6)
|
||||
expect(TypeUtils.getNounIndex(NounType.Document)).toBe(13)
|
||||
})
|
||||
|
||||
test('should return correct index for resource', () => {
|
||||
expect(TypeUtils.getNounIndex(NounType.Resource)).toBe(30)
|
||||
expect(TypeUtils.getNounIndex(NounType.Resource)).toBe(34)
|
||||
})
|
||||
|
||||
test('should work for all noun types', () => {
|
||||
const allTypes = Object.values(NounType)
|
||||
expect(allTypes).toHaveLength(31)
|
||||
expect(allTypes).toHaveLength(42)
|
||||
|
||||
for (const type of allTypes) {
|
||||
const index = TypeUtils.getNounIndex(type)
|
||||
expect(index).toBeGreaterThanOrEqual(0)
|
||||
expect(index).toBeLessThanOrEqual(30)
|
||||
expect(index).toBeLessThanOrEqual(41)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('TypeUtils.getVerbIndex', () => {
|
||||
test('should return correct index for relatedTo', () => {
|
||||
expect(TypeUtils.getVerbIndex(VerbType.RelatedTo)).toBe(0)
|
||||
expect(TypeUtils.getVerbIndex(VerbType.RelatedTo)).toBe(3)
|
||||
})
|
||||
|
||||
test('should return correct index for creates', () => {
|
||||
expect(TypeUtils.getVerbIndex(VerbType.Creates)).toBe(10)
|
||||
expect(TypeUtils.getVerbIndex(VerbType.Creates)).toBe(17)
|
||||
})
|
||||
|
||||
test('should return correct index for competes', () => {
|
||||
expect(TypeUtils.getVerbIndex(VerbType.Competes)).toBe(39)
|
||||
expect(TypeUtils.getVerbIndex(VerbType.Competes)).toBe(50)
|
||||
})
|
||||
|
||||
test('should work for all verb types', () => {
|
||||
const allTypes = Object.values(VerbType)
|
||||
expect(allTypes).toHaveLength(40)
|
||||
expect(allTypes).toHaveLength(127)
|
||||
|
||||
for (const type of allTypes) {
|
||||
const index = TypeUtils.getVerbIndex(type)
|
||||
expect(index).toBeGreaterThanOrEqual(0)
|
||||
expect(index).toBeLessThanOrEqual(39)
|
||||
expect(index).toBeLessThanOrEqual(126)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
|
@ -214,8 +214,8 @@ describe('Type System Foundation', () => {
|
|||
|
||||
expect(entityCountsByType[0]).toBe(1000) // person
|
||||
expect(entityCountsByType[6]).toBe(500) // document
|
||||
expect(entityCountsByType.length).toBe(31)
|
||||
expect(entityCountsByType.byteLength).toBe(124) // 31 × 4 bytes
|
||||
expect(entityCountsByType.length).toBe(42)
|
||||
expect(entityCountsByType.byteLength).toBe(168) // 42 × 4 bytes
|
||||
})
|
||||
|
||||
test('should enable O(1) verb tracking with Uint32Array', () => {
|
||||
|
|
@ -227,8 +227,8 @@ describe('Type System Foundation', () => {
|
|||
|
||||
expect(verbCountsByType[0]).toBe(5000) // relatedTo
|
||||
expect(verbCountsByType[10]).toBe(2000) // creates
|
||||
expect(verbCountsByType.length).toBe(40)
|
||||
expect(verbCountsByType.byteLength).toBe(160) // 40 × 4 bytes
|
||||
expect(verbCountsByType.length).toBe(127)
|
||||
expect(verbCountsByType.byteLength).toBe(508) // 127 × 4 bytes
|
||||
})
|
||||
})
|
||||
|
||||
|
|
@ -238,7 +238,7 @@ describe('Type System Foundation', () => {
|
|||
const verbCounts = new Uint32Array(VERB_TYPE_COUNT)
|
||||
|
||||
const totalBytes = entityCounts.byteLength + verbCounts.byteLength
|
||||
expect(totalBytes).toBe(284) // 124 + 160 = 284 bytes (vs ~60KB with Maps)
|
||||
expect(totalBytes).toBe(676) // 168 + 508 = 676 bytes (vs ~60KB with Maps)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -35,8 +35,8 @@ describe('MetadataIndexManager - Phase 1b: Type-Aware Features', () => {
|
|||
expect(managerAny.verbCountsByTypeFixed.length).toBe(VERB_TYPE_COUNT)
|
||||
})
|
||||
|
||||
it('should have 99.76% memory reduction vs Maps', () => {
|
||||
// Fixed-size arrays: 31 × 4 bytes + 40 × 4 bytes = 284 bytes
|
||||
it('should have 99.44% memory reduction vs Maps', () => {
|
||||
// Fixed-size arrays: 42 × 4 bytes + 127 × 4 bytes = 676 bytes
|
||||
const fixedSize = (NOUN_TYPE_COUNT + VERB_TYPE_COUNT) * 4
|
||||
|
||||
// Map overhead: ~120KB for string keys, pointers, hash table
|
||||
|
|
@ -44,8 +44,8 @@ describe('MetadataIndexManager - Phase 1b: Type-Aware Features', () => {
|
|||
|
||||
const reduction = ((mapSize - fixedSize) / mapSize) * 100
|
||||
|
||||
expect(fixedSize).toBe(284)
|
||||
expect(reduction).toBeGreaterThan(99.7)
|
||||
expect(fixedSize).toBe(676)
|
||||
expect(reduction).toBeGreaterThan(99.4)
|
||||
})
|
||||
|
||||
it('should track entity counts in Uint32Arrays when adding entities', async () => {
|
||||
|
|
@ -309,8 +309,8 @@ describe('MetadataIndexManager - Phase 1b: Type-Aware Features', () => {
|
|||
const verbArraySize = managerAny.verbCountsByTypeFixed.byteLength
|
||||
const totalFixedSize = nounArraySize + verbArraySize
|
||||
|
||||
// Should be exactly 284 bytes
|
||||
expect(totalFixedSize).toBe(284)
|
||||
// Should be exactly 676 bytes
|
||||
expect(totalFixedSize).toBe(676)
|
||||
})
|
||||
})
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue