brainy/package.json
David Snelling 92c96246fb feat(v4.0.0): Complete metadata/vector separation architecture with Azure support
This commit completes the core v4.0.0 architecture changes for billion-scale
performance with metadata/vector separation. NO RELEASE YET - remaining optimizations
and testing required before production release.

## Core v4.0.0 Architecture Changes

### Type System Updates
- Fixed all TypeScript compilation errors (zero errors achieved)
- Updated HNSWNoun/HNSWVerb to separate core fields from metadata
- Implemented HNSWNounWithMetadata/HNSWVerbWithMetadata for API boundaries
- Added required 'noun' field to NounMetadata for semantic structure
- Renamed verb.type to verb.verb for consistency

### Storage Adapter Updates
**All adapters updated for v4.0.0 two-file storage pattern:**
- memoryStorage: Proper metadata/vector separation
- fileSystemStorage: Two-file pattern with sharding
- opfsStorage: Browser persistent storage updated
- s3CompatibleStorage: AWS/MinIO/DigitalOcean support
- r2Storage: Cloudflare R2 optimization
- gcsStorage: Google Cloud with ADC support
- **azureBlobStorage: NEW - Full Azure Blob Storage support**

### Storage Features
- BaseStorage: Internal vs public method separation (_getNoun vs getNoun)
- Two-file storage: Vectors in one file, metadata in another
- Change tracking: getChangesSince return type updated
- Pagination: getNounsWithPagination returns WithMetadata types

### Azure Blob Storage Integration (NEW)
- Native @azure/storage-blob SDK integration
- Four authentication methods:
  * DefaultAzureCredential (Managed Identity) - recommended
  * Connection String - simplest setup
  * Account Name + Key - traditional auth
  * SAS Token - delegated access
- High-volume mode with write buffering
- Adaptive backpressure for throttling
- UUID-based sharding for billion-scale
- Full HNSW support with graph persistence

### Utility Updates
- EmbeddingManager: Updated to accept Record<string, unknown>
- LSMTree: Wrapped data in NounMetadata structure with 'noun' field
- EntityIdMapper: Fixed nested metadata.data structure access
- MetadataIndex: Fixed field type inference integration
- PeriodicCleanup: Updated for new metadata structure

### Core API Updates
- Brainy: Updated verb property access from v.type to v.verb
- ConfigAPI: Fixed NounMetadata access patterns
- DataAPI: Updated metadata handling

### Documentation Updates
- CREATING-AUGMENTATIONS.md: v4.0.0 breaking changes guide
- DEVELOPER-GUIDE.md: Migration checklist and examples
- COMPLETE-REFERENCE.md: v4.0.0 architecture improvements
- **finite-type-system.md: NEW - Revolutionary type system benefits**

### Build & Dependencies
- Zero TypeScript compilation errors
- Added @azure/storage-blob and @azure/identity
- 591 tests passing (23 timeout in long-running neural tests)

## What's NOT in This Release
This is a work-in-progress commit. Before v4.0.0 release we need:
- Storage adapter optimizations (batch operations, compression)
- Azure blob tier management (Hot/Cool/Archive)
- Cost optimization implementations
- Additional performance testing at billion-scale
- Migration guides for v3.x users

## Testing
- Clean build: 
- Type checking:  (zero errors)
- Test suite:  (591/614 passing, timeouts in neural tests only)

🔐 Generated with Claude Code
https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 12:29:27 -07:00

227 lines
7.5 KiB
JSON
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"name": "@soulcraft/brainy",
"version": "3.50.2",
"description": "Universal Knowledge Protocol™ - World's first Triple Intelligence database unifying vector, graph, and document search in one API. 31 nouns × 40 verbs for infinite expressiveness.",
"main": "dist/index.js",
"module": "dist/index.js",
"types": "dist/index.d.ts",
"type": "module",
"bin": {
"brainy": "bin/brainy.js"
},
"sideEffects": [
"./dist/setup.js",
"./dist/utils/textEncoding.js",
"./src/setup.ts",
"./src/utils/textEncoding.ts"
],
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
},
"./setup": {
"import": "./dist/setup.js",
"types": "./dist/setup.d.ts"
},
"./types/graphTypes": {
"import": "./dist/types/graphTypes.js",
"types": "./dist/types/graphTypes.d.ts"
},
"./types/augmentations": {
"import": "./dist/types/augmentations.js",
"types": "./dist/types/augmentations.d.ts"
},
"./utils/textEncoding": {
"import": "./dist/utils/textEncoding.js",
"types": "./dist/utils/textEncoding.d.ts"
},
"./universal": {
"import": "./dist/universal/index.js",
"types": "./dist/universal/index.d.ts"
}
},
"browser": {
"./dist/distributed": false,
"./dist/cli": false,
"./dist/scripts": false,
"./dist/cortex": false,
"./dist/cortex/neuralImport.js": false,
"./dist/cortex/performanceMonitor.js": false,
"./dist/cortex/healthCheck.js": false,
"./dist/cortex/backupRestore.js": false
},
"engines": {
"node": "22.x"
},
"scripts": {
"build": "npm run build:types:if-needed && npm run build:patterns:if-needed && npm run build:keywords:if-needed && tsc && tsc -p tsconfig.cli.json",
"build:types": "tsx scripts/buildTypeEmbeddings.ts",
"build:types:if-needed": "node scripts/check-type-embeddings.cjs || npm run build:types",
"build:types:force": "npm run build:types",
"build:patterns": "tsx scripts/buildEmbeddedPatterns.ts",
"build:patterns:if-needed": "node scripts/check-patterns.cjs || npm run build:patterns",
"build:patterns:force": "npm run build:patterns",
"build:keywords": "tsx scripts/buildKeywordEmbeddings.ts",
"build:keywords:if-needed": "node scripts/check-keyword-embeddings.cjs || npm run build:keywords",
"build:keywords:force": "npm run build:keywords",
"prepare": "npm run build",
"test": "npm run test:unit",
"test:watch": "NODE_OPTIONS='--max-old-space-size=8192' vitest --config tests/configs/vitest.unit.config.ts",
"test:coverage": "NODE_OPTIONS='--max-old-space-size=8192' vitest run --config tests/configs/vitest.unit.config.ts --coverage",
"test:unit": "NODE_OPTIONS='--max-old-space-size=8192' vitest run --config tests/configs/vitest.unit.config.ts",
"test:integration": "NODE_OPTIONS='--max-old-space-size=32768' vitest run --config tests/configs/vitest.integration.config.ts",
"test:s3": "vitest run tests/integration/s3-storage.test.ts",
"test:distributed": "vitest run tests/integration/distributed.test.ts",
"test:cloud": "npm run test:s3 && npm run test:distributed",
"test:all": "npm run test:unit && npm run test:integration",
"test:ci-unit": "CI=true vitest run --config tests/configs/vitest.unit.config.ts",
"test:ci-integration": "NODE_OPTIONS='--max-old-space-size=16384' CI=true vitest run --config tests/configs/vitest.integration.config.ts",
"test:ci": "npm run test:ci-unit",
"download-models": "node scripts/download-models.cjs",
"download-models:q8": "node scripts/download-models.cjs",
"models:verify": "node scripts/ensure-models.js",
"lint": "eslint --ext .ts,.js src/",
"lint:fix": "eslint --ext .ts,.js src/ --fix",
"format": "prettier --write \"src/**/*.{ts,js}\"",
"format:check": "prettier --check \"src/**/*.{ts,js}\"",
"migrate:logger": "tsx scripts/migrate-to-structured-logger.ts",
"migrate:logger:dry": "tsx scripts/migrate-to-structured-logger.ts --dry-run",
"release": "./scripts/release.sh patch",
"release:patch": "./scripts/release.sh patch",
"release:minor": "./scripts/release.sh minor",
"release:major": "./scripts/release.sh major",
"release:dry": "./scripts/release.sh patch --dry-run",
"release:standard-version": "standard-version",
"release:standard-version:patch": "standard-version --release-as patch",
"release:standard-version:minor": "standard-version --release-as minor",
"release:standard-version:major": "standard-version --release-as major"
},
"keywords": [
"ai-database",
"vector-database",
"graph-database",
"field-filtering",
"triple-intelligence",
"hnsw",
"embeddings",
"semantic-search",
"machine-learning",
"artificial-intelligence",
"data-storage",
"indexing",
"typescript"
],
"author": "Brainy Contributors",
"license": "MIT",
"private": false,
"publishConfig": {
"access": "public"
},
"homepage": "https://github.com/soulcraftlabs/brainy",
"bugs": {
"url": "https://github.com/soulcraftlabs/brainy/issues"
},
"repository": {
"type": "git",
"url": "git+https://github.com/soulcraftlabs/brainy.git"
},
"files": [
"dist/**/*.js",
"dist/**/*.d.ts",
"bin/",
"scripts/download-models.cjs",
"scripts/ensure-models.js",
"scripts/prepare-models.js",
"brainy.png",
"LICENSE",
"README.md",
"CHANGELOG.md"
],
"overrides": {
"boolean": "3.2.0"
},
"devDependencies": {
"@rollup/plugin-commonjs": "^28.0.6",
"@rollup/plugin-node-resolve": "^16.0.1",
"@rollup/plugin-replace": "^6.0.2",
"@rollup/plugin-terser": "^0.4.4",
"@testcontainers/redis": "^11.5.1",
"@types/node": "^20.11.30",
"@types/uuid": "^10.0.0",
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.0.0",
"@typescript-eslint/parser": "^8.0.0",
"@vitest/coverage-v8": "^3.2.4",
"jspdf": "^3.0.3",
"minio": "^8.0.5",
"standard-version": "^9.5.0",
"testcontainers": "^11.5.1",
"tsx": "^4.19.2",
"typescript": "^5.4.5",
"vitest": "^3.2.4"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.540.0",
"@azure/identity": "^4.0.0",
"@azure/storage-blob": "^12.17.0",
"@google-cloud/storage": "^7.14.0",
"@huggingface/transformers": "^3.7.2",
"@msgpack/msgpack": "^3.1.2",
"boxen": "^8.0.1",
"chalk": "^5.3.0",
"chardet": "^2.0.0",
"cli-table3": "^0.6.5",
"commander": "^11.1.0",
"csv-parse": "^6.1.0",
"inquirer": "^12.9.3",
"ora": "^8.2.0",
"pdfjs-dist": "^4.0.379",
"prompts": "^2.4.2",
"roaring-wasm": "^1.1.0",
"uuid": "^9.0.1",
"ws": "^8.18.3",
"xlsx": "^0.18.5"
},
"prettier": {
"arrowParens": "always",
"bracketSameLine": true,
"bracketSpacing": true,
"htmlWhitespaceSensitivity": "css",
"printWidth": 80,
"proseWrap": "preserve",
"semi": false,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "none",
"useTabs": false
},
"eslintConfig": {
"root": true,
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended"
],
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint"
],
"rules": {
"@typescript-eslint/no-explicit-any": "off",
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
"warn",
{
"args": "after-used",
"argsIgnorePattern": "^_"
}
],
"semi": "off",
"@typescript-eslint/semi": [
"error",
"never"
],
"no-extra-semi": "off"
}
}
}