refactor(8.0)!: remove distributed clustering subsystem — inert/orphaned, scale is single-process + native provider
The distributed-clustering subsystem never ran in production: it was inert, orphaned dead code (faked consensus, stub replication, no live wiring, and it did not interoperate with the 8.0 Db API). Brainy 8.0 is a single-process library. Scale is single-process + the optional native provider (@soulcraft/cortex, on-disk DiskANN to 10B+ vectors) + per-tenant pools + horizontal read scaling (many reader processes, one writer). Removed: - src/distributed/ entirely (coordinator, shardManager, cacheSync, readWriteSeparation, queryPlanner, healthMonitor, configManager, hashPartitioner, shardMigration, domainDetector, storageDiscovery, http/network transports). ReaderMode/HybridMode relocated to src/storage/operationalModes.ts (slimmed to the live surface). - src/types/distributedTypes.ts; config.distributed field + JSDoc; coreTypes distributedConfig; memoryStorage distributedConfig persistence. - DistributedRole enum + src/config/distributedPresets.ts and the orphaned src/config/extensibleConfig.ts (config/augmentation registry built on removed cloud adapters + distributed presets), plus their src/index.ts re-exports. - 13 BRAINY_* cluster env vars; the storage setDistributedComponents hook; enableDistributedSearch (dead config flag); the metadata partition field; the distributed_ reserved key prefix. - Orphaned src/storage/readOnlyOptimizations.ts (zero importers). - Tests targeting the subsystem: distributed-demo, distributed-cluster helper, distributed-transactions, sharding-transactions. - Docs: EXTENDING_STORAGE.md (deleted); scrubbed distributed/cluster/Raft/ shard-manager/multi-node prose from v3-features, enterprise-for-everyone, augmentations-actual, complete-feature-list, vfs/README, vfs/ROADMAP, vfs/VFS_CORE, capacity-planning, transactions, MIGRATION-V3-TO-V4, storage-architecture; reframed scale prose to the 8.0 model. Kept: src/storage/sharding.ts (local-disk 256-bucket directory sharding via getShardIdFromUuid — used live by baseStorage, unrelated to clustering); the multi-process mode: 'reader' | 'writer' roles; semantic/HNSW clustering. RELEASES.md: added a removed-surfaces row documenting the cut and the 8.0 scale model.
This commit is contained in:
parent
f8e0079d3f
commit
00d3203d68
51 changed files with 153 additions and 9889 deletions
|
|
@ -165,32 +165,28 @@ await brain.syncWith({
|
|||
- **Webhook support**: React to changes
|
||||
- **API generation**: Auto-generate REST/GraphQL APIs
|
||||
|
||||
### 🌍 Enterprise Scale 🚧 Coming Soon
|
||||
### 🌍 Scale
|
||||
|
||||
**Everyone gets planetary scale:**
|
||||
**Everyone gets the same scale model:**
|
||||
|
||||
```typescript
|
||||
// Same architecture Netflix uses, free for you
|
||||
const brain = new Brainy({
|
||||
clustering: {
|
||||
enabled: true, // Distributed mode
|
||||
sharding: 'automatic', // Auto-sharding
|
||||
replication: 3, // Triple replication
|
||||
consensus: 'raft', // Strong consistency
|
||||
geoDistribution: true // Multi-region support
|
||||
}
|
||||
})
|
||||
// Pure JS by default; install the optional native provider for billions of vectors
|
||||
const brain = new Brainy()
|
||||
|
||||
// Handles everything from 1 to 1 billion entities
|
||||
// 1 → ~1M vectors: pure-JS HNSW, zero extra setup
|
||||
// 1M → 10B+ vectors: install @soulcraft/cortex for the native DiskANN provider
|
||||
```
|
||||
|
||||
**Scaling features:**
|
||||
- **Horizontal scaling**: Add nodes as needed
|
||||
- **Auto-sharding**: Distributes data automatically
|
||||
- **Multi-region**: Global distribution
|
||||
- **Load balancing**: Automatic request distribution
|
||||
- **Zero-downtime upgrades**: Rolling updates
|
||||
- **Infinite scale**: No upper limits
|
||||
**Scaling model:**
|
||||
- **Single process, no cluster**: Brainy runs in one process — no coordinator,
|
||||
no peer discovery, no consensus to operate
|
||||
- **Optional native provider**: install `@soulcraft/cortex` to back the index with
|
||||
on-disk DiskANN that scales to 10B+ vectors on one machine
|
||||
- **Per-tenant pools**: isolate tenants by giving each its own Brainy instance and
|
||||
storage directory
|
||||
- **Horizontal read scaling**: run many reader processes against one shared on-disk
|
||||
store (single writer, many readers); replicate the artifact with your operator
|
||||
tooling
|
||||
|
||||
### 🛡️ Enterprise Compliance 🚧 Coming Soon
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue