chore(8.0): Phase A + B — purge all @deprecated APIs + cacheManager dead branches
PHASE A — every @deprecated marker resolved (~25 removed) src/coreTypes.ts - GraphVerb: dropped the "@deprecated Will be replaced by HNSWVerbWithMetadata" note. GraphVerb IS the canonical contract — every public API path speaks it. Removed the `source` and `target` legacy alias fields (renamed `from` / `to` callers years ago; no consumers remain). - StorageAdapter: dropped the "@deprecated Use getNouns() with filter" notes from `getNounsByNounType`, `getVerbsBySource`, `getVerbsByTarget`, `getVerbsByType`. They were never deprecated in spirit — they're useful non-paginated convenience wrappers over the paginated `getNouns()` / `getVerbs()` surface. Refreshed JSDoc to explain the role. src/types/graphTypes.ts - Mirrored the GraphVerb cleanup: dropped `source` + `target` legacy aliases. sourceId + targetId are the canonical fields. src/import/ImportCoordinator.ts - Deleted the entire DeprecatedImportOptions interface block (130 LOC). It was a v3 → v4 migration tool using the `?: never` trick to force compile errors on dropped options. Five major versions in, the forced-error gate is no longer pulling its weight. src/triple/TripleIntelligence.ts - Deleted `TripleIntelligenceEngine = any` alias. No consumers; superseded by `TripleIntelligenceSystem`. src/storage/cow/binaryDataCodec.ts - Deleted `wrapBinaryData()`. The COW dispatch layer in `baseStorage.ts` routes by key-prefix convention; the old guess-by-JSON-parse codec was fragile (compressed bytes can accidentally parse as JSON) and unused. src/storage/baseStorage.ts - Refreshed JSDoc on `convertHNSWVerbToGraphVerb()` — the method is alive and used internally; the deprecation note was stale. src/embeddings/wasm/AssetLoader.ts → DELETED - File was @deprecated since model weights moved into the Candle WASM bundle. No consumers. Removed from `embeddings/wasm/index.ts` exports. src/embeddings/wasm/types.ts - Dropped @deprecated tags on `TokenizerConfig` + `TokenizedInput` — still used by `WordPieceTokenizer` (auxiliary tokenization). Deleted `InferenceConfig` (truly dead). Updated `embeddings/wasm/index.ts` exports. src/utils/metadataIndex.ts - Deleted `getIdsForCriteria()` — pure alias for `getIdsForFilter()`, no consumers. src/interfaces/IIndex.ts - Removed RebuildOptions.lazy (deprecated and unused; lazy mode is auto- selected by available-memory detection). src/hnsw/hnswIndex.ts - Removed `getNouns()` (returned a full Map; deprecated in favor of pagination years ago and no consumers in src/ or tests/). PHASE B — cacheManager dead StorageType branches src/storage/cacheManager.ts - Collapsed the `isRemoteStorage` flag and its 15 dead conditional branches spanning calculateOptimalCacheSize() and calculateOptimalBatchSize(). After dropping cloud adapters in step 7, `coldStorageType` is never S3 or REMOTE_API; the branches were dead. Cache sizing and batch sizing now honor the filesystem-only reality with simpler heuristics. - Collapsed `detectWarmStorageType()` + `detectColdStorageType()` from ~40 LOC of environment-+-availability branching to 2-line returns of `StorageType.FILESYSTEM`. Brainy 8.0 ships filesystem + memory only. NOT YET — Phases C-G in follow-up commits C: storageAutoConfig.ts + zeroConfig + extensibleConfig + sharedConfigManager D: TODO/FIXME sweep across src/ E: skipped tests + the parallel-test race condition F: docs deep clean (BATCHING, augmentations, READMEs) G: browser support drop (the last 2 @deprecated) VERIFICATION - npx tsc --noEmit: clean - npm test: 1408 / 1409 (same pre-existing race-condition outstanding)
This commit is contained in:
parent
9f9a41599e
commit
cb16a39a0c
15 changed files with 98 additions and 718 deletions
|
|
@ -1,266 +0,0 @@
|
|||
/**
|
||||
* Asset Loader
|
||||
*
|
||||
* @deprecated This class is no longer used. Model weights are now embedded
|
||||
* in the Candle WASM binary at compile time. Kept for backward compatibility.
|
||||
*
|
||||
* Previously: Resolved paths to ONNX model files across environments.
|
||||
* Now: Use CandleEmbeddingEngine which loads embedded model automatically.
|
||||
*/
|
||||
|
||||
import { MODEL_CONSTANTS } from './types.js'
|
||||
|
||||
// Cache resolved paths
|
||||
let cachedModelDir: string | null = null
|
||||
let cachedVocab: Record<string, number> | null = null
|
||||
|
||||
/**
|
||||
* Asset loader for model files
|
||||
*/
|
||||
export class AssetLoader {
|
||||
private modelDir: string | null = null
|
||||
|
||||
/**
|
||||
* Get the model directory path
|
||||
*/
|
||||
async getModelDir(): Promise<string> {
|
||||
if (this.modelDir) {
|
||||
return this.modelDir
|
||||
}
|
||||
|
||||
if (cachedModelDir) {
|
||||
this.modelDir = cachedModelDir
|
||||
return cachedModelDir
|
||||
}
|
||||
|
||||
// Try to resolve model directory
|
||||
const resolved = await this.resolveModelDir()
|
||||
this.modelDir = resolved
|
||||
cachedModelDir = resolved
|
||||
return resolved
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the model directory across environments
|
||||
*/
|
||||
private async resolveModelDir(): Promise<string> {
|
||||
// 1. Check environment variable
|
||||
if (typeof process !== 'undefined' && process.env?.BRAINY_MODEL_PATH) {
|
||||
const envPath = process.env.BRAINY_MODEL_PATH
|
||||
if (await this.pathExists(envPath)) {
|
||||
return envPath
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Try common locations
|
||||
const modelName = MODEL_CONSTANTS.MODEL_NAME + '-q8'
|
||||
const possiblePaths = [
|
||||
// Package assets (when installed as dependency)
|
||||
`./assets/models/${modelName}`,
|
||||
`./node_modules/@soulcraft/brainy/assets/models/${modelName}`,
|
||||
// Development paths
|
||||
`../assets/models/${modelName}`,
|
||||
// Absolute from package root
|
||||
this.getPackageRootPath(`assets/models/${modelName}`),
|
||||
].filter(Boolean) as string[]
|
||||
|
||||
for (const path of possiblePaths) {
|
||||
if (await this.pathExists(path)) {
|
||||
return path
|
||||
}
|
||||
}
|
||||
|
||||
// If no path found, return default (will error on use)
|
||||
return `./assets/models/${modelName}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Get package root path (Node.js/Bun only)
|
||||
*/
|
||||
private getPackageRootPath(relativePath: string): string | null {
|
||||
if (typeof process === 'undefined') {
|
||||
return null
|
||||
}
|
||||
|
||||
try {
|
||||
// Use __dirname equivalent
|
||||
const url = new URL(import.meta.url)
|
||||
const currentDir = url.pathname.replace(/\/[^/]*$/, '')
|
||||
// Go up from src/embeddings/wasm to package root
|
||||
const packageRoot = currentDir.replace(/\/src\/embeddings\/wasm$/, '')
|
||||
return `${packageRoot}/${relativePath}`
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if path exists (works in Node.js/Bun)
|
||||
*/
|
||||
private async pathExists(path: string): Promise<boolean> {
|
||||
if (typeof process === 'undefined') {
|
||||
// Browser - check via fetch
|
||||
try {
|
||||
const response = await fetch(path, { method: 'HEAD' })
|
||||
return response.ok
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// Node.js/Bun
|
||||
try {
|
||||
const fs = await import('node:fs/promises')
|
||||
await fs.access(path)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get path to ONNX model file
|
||||
*/
|
||||
async getModelPath(): Promise<string> {
|
||||
const dir = await this.getModelDir()
|
||||
return `${dir}/model.onnx`
|
||||
}
|
||||
|
||||
/**
|
||||
* Get path to vocabulary file
|
||||
*/
|
||||
async getVocabPath(): Promise<string> {
|
||||
const dir = await this.getModelDir()
|
||||
return `${dir}/vocab.json`
|
||||
}
|
||||
|
||||
/**
|
||||
* Load vocabulary from JSON file
|
||||
*/
|
||||
async loadVocab(): Promise<Record<string, number>> {
|
||||
if (cachedVocab) {
|
||||
return cachedVocab
|
||||
}
|
||||
|
||||
const vocabPath = await this.getVocabPath()
|
||||
|
||||
if (typeof process !== 'undefined') {
|
||||
// Node.js/Bun - read from filesystem
|
||||
try {
|
||||
const fs = await import('node:fs/promises')
|
||||
const content = await fs.readFile(vocabPath, 'utf-8')
|
||||
cachedVocab = JSON.parse(content)
|
||||
return cachedVocab!
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`Failed to load vocabulary from ${vocabPath}: ${error instanceof Error ? error.message : String(error)}`
|
||||
)
|
||||
}
|
||||
} else {
|
||||
// Browser - fetch
|
||||
try {
|
||||
const response = await fetch(vocabPath)
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}`)
|
||||
}
|
||||
cachedVocab = await response.json()
|
||||
return cachedVocab!
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`Failed to fetch vocabulary from ${vocabPath}: ${error instanceof Error ? error.message : String(error)}`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load model as ArrayBuffer (for ONNX session)
|
||||
*/
|
||||
async loadModel(): Promise<ArrayBuffer> {
|
||||
const modelPath = await this.getModelPath()
|
||||
|
||||
if (typeof process !== 'undefined') {
|
||||
// Node.js/Bun - read from filesystem
|
||||
try {
|
||||
const fs = await import('node:fs/promises')
|
||||
const buffer = await fs.readFile(modelPath)
|
||||
// Convert Node.js Buffer to ArrayBuffer
|
||||
return new Uint8Array(buffer).buffer as ArrayBuffer
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`Failed to load model from ${modelPath}: ${error instanceof Error ? error.message : String(error)}`
|
||||
)
|
||||
}
|
||||
} else {
|
||||
// Browser - fetch
|
||||
try {
|
||||
const response = await fetch(modelPath)
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}`)
|
||||
}
|
||||
return await response.arrayBuffer()
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`Failed to fetch model from ${modelPath}: ${error instanceof Error ? error.message : String(error)}`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify all required assets exist
|
||||
*/
|
||||
async verifyAssets(): Promise<{
|
||||
valid: boolean
|
||||
modelPath: string
|
||||
vocabPath: string
|
||||
errors: string[]
|
||||
}> {
|
||||
const errors: string[] = []
|
||||
const modelPath = await this.getModelPath()
|
||||
const vocabPath = await this.getVocabPath()
|
||||
|
||||
if (!(await this.pathExists(modelPath))) {
|
||||
errors.push(`Model file not found: ${modelPath}`)
|
||||
}
|
||||
|
||||
if (!(await this.pathExists(vocabPath))) {
|
||||
errors.push(`Vocabulary file not found: ${vocabPath}`)
|
||||
}
|
||||
|
||||
return {
|
||||
valid: errors.length === 0,
|
||||
modelPath,
|
||||
vocabPath,
|
||||
errors,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear cached paths (for testing)
|
||||
*/
|
||||
clearCache(): void {
|
||||
this.modelDir = null
|
||||
cachedModelDir = null
|
||||
cachedVocab = null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create asset loader instance
|
||||
*/
|
||||
export function createAssetLoader(): AssetLoader {
|
||||
return new AssetLoader()
|
||||
}
|
||||
|
||||
/**
|
||||
* Singleton asset loader
|
||||
*/
|
||||
let singletonLoader: AssetLoader | null = null
|
||||
|
||||
export function getAssetLoader(): AssetLoader {
|
||||
if (!singletonLoader) {
|
||||
singletonLoader = new AssetLoader()
|
||||
}
|
||||
return singletonLoader
|
||||
}
|
||||
|
|
@ -25,16 +25,14 @@ export {
|
|||
cosineSimilarity,
|
||||
} from './CandleEmbeddingEngine.js'
|
||||
|
||||
// Legacy components (for backward compatibility - not needed with Candle)
|
||||
// Auxiliary embedding components
|
||||
export { WordPieceTokenizer, createTokenizer } from './WordPieceTokenizer.js'
|
||||
export { EmbeddingPostProcessor, createPostProcessor } from './EmbeddingPostProcessor.js'
|
||||
export { AssetLoader, getAssetLoader, createAssetLoader } from './AssetLoader.js'
|
||||
|
||||
// Types
|
||||
export type {
|
||||
TokenizerConfig,
|
||||
TokenizedInput,
|
||||
InferenceConfig,
|
||||
EmbeddingResult,
|
||||
EngineStats,
|
||||
ModelConfig,
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@
|
|||
*/
|
||||
|
||||
/**
|
||||
* Tokenizer configuration for WordPiece
|
||||
* @deprecated Tokenization now happens in Rust/WASM - kept for backward compatibility
|
||||
* Tokenizer configuration for WordPiece. Used by `WordPieceTokenizer` for the
|
||||
* JS-side tokenization path (auxiliary to the Rust/WASM embedding engine).
|
||||
*/
|
||||
export interface TokenizerConfig {
|
||||
/** Vocabulary mapping word → token ID */
|
||||
|
|
@ -27,8 +27,7 @@ export interface TokenizerConfig {
|
|||
}
|
||||
|
||||
/**
|
||||
* Result of tokenization
|
||||
* @deprecated Tokenization now happens in Rust/WASM - kept for backward compatibility
|
||||
* Result of tokenization. Returned by `WordPieceTokenizer.encode()`.
|
||||
*/
|
||||
export interface TokenizedInput {
|
||||
/** Token IDs including [CLS] and [SEP] */
|
||||
|
|
@ -41,23 +40,6 @@ export interface TokenizedInput {
|
|||
tokenCount: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Inference engine configuration
|
||||
* @deprecated Model is now embedded in WASM - kept for backward compatibility
|
||||
*/
|
||||
export interface InferenceConfig {
|
||||
/** Path to model file (not used with embedded model) */
|
||||
modelPath: string
|
||||
/** Path to WASM files directory */
|
||||
wasmPath?: string
|
||||
/** Number of threads (1 for universal compatibility) */
|
||||
numThreads: number
|
||||
/** Enable SIMD if available */
|
||||
enableSimd: boolean
|
||||
/** Enable CPU memory arena */
|
||||
enableCpuMemArena: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Embedding result with metadata
|
||||
*/
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue