perf: pre-compute type embeddings at build time (zero runtime cost)

Browse source

Major optimization - all type embeddings now built into package:

Build-time generation:
- Created scripts/buildTypeEmbeddings.ts to generate all type embeddings
- Generates embeddings for 31 NounTypes + 40 VerbTypes at build time
- Stores as base64-encoded binary data in embeddedTypeEmbeddings.ts
- Added check script to rebuild only when needed

Updated all consumers:
- NeuralEntityExtractor: loads pre-computed embeddings (instant)
- BrainyTypes: loads pre-computed embeddings (instant init)
- NaturalLanguageProcessor: loads pre-computed embeddings (instant init)

Build process:
- Added npm run build:types to generate embeddings
- Added npm run build:types:if-needed for conditional rebuild
- Integrated into main build pipeline
- Auto-rebuilds only when types or build script change

Benefits:
- Zero runtime cost - embeddings loaded instantly
- Survives all container restarts
- All 71 types always available (31 nouns + 40 verbs)
- ~100KB memory overhead for permanent performance gain
- Eliminates 5-10 second initialization delay

This completes the type embedding optimization started in v3.32.5

...

This commit is contained in:

David Snelling 2025-10-09 18:08:57 -07:00

parent 87eb60d527

commit 0d649b8a79

9 changed files with 643 additions and 111 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

117

src/neural/embeddedTypeEmbeddings.ts Normal file

View file

File diff suppressed because one or more lines are too long