refactor(8.0)!: remove distributed clustering subsystem — inert/orphaned, scale is single-process + native provider

The distributed-clustering subsystem never ran in production: it was inert,
orphaned dead code (faked consensus, stub replication, no live wiring, and it
did not interoperate with the 8.0 Db API). Brainy 8.0 is a single-process
library. Scale is single-process + the optional native provider
(@soulcraft/cortex, on-disk DiskANN to 10B+ vectors) + per-tenant pools +
horizontal read scaling (many reader processes, one writer).

Removed:
- src/distributed/ entirely (coordinator, shardManager, cacheSync,
  readWriteSeparation, queryPlanner, healthMonitor, configManager,
  hashPartitioner, shardMigration, domainDetector, storageDiscovery, http/network
  transports). ReaderMode/HybridMode relocated to src/storage/operationalModes.ts
  (slimmed to the live surface).
- src/types/distributedTypes.ts; config.distributed field + JSDoc;
  coreTypes distributedConfig; memoryStorage distributedConfig persistence.
- DistributedRole enum + src/config/distributedPresets.ts and the orphaned
  src/config/extensibleConfig.ts (config/augmentation registry built on removed
  cloud adapters + distributed presets), plus their src/index.ts re-exports.
- 13 BRAINY_* cluster env vars; the storage setDistributedComponents hook;
  enableDistributedSearch (dead config flag); the metadata partition field;
  the distributed_ reserved key prefix.
- Orphaned src/storage/readOnlyOptimizations.ts (zero importers).
- Tests targeting the subsystem: distributed-demo, distributed-cluster helper,
  distributed-transactions, sharding-transactions.
- Docs: EXTENDING_STORAGE.md (deleted); scrubbed distributed/cluster/Raft/
  shard-manager/multi-node prose from v3-features, enterprise-for-everyone,
  augmentations-actual, complete-feature-list, vfs/README, vfs/ROADMAP,
  vfs/VFS_CORE, capacity-planning, transactions, MIGRATION-V3-TO-V4,
  storage-architecture; reframed scale prose to the 8.0 model.

Kept: src/storage/sharding.ts (local-disk 256-bucket directory sharding via
getShardIdFromUuid — used live by baseStorage, unrelated to clustering);
the multi-process mode: 'reader' | 'writer' roles; semantic/HNSW clustering.

RELEASES.md: added a removed-surfaces row documenting the cut and the 8.0
scale model.
This commit is contained in:
David Snelling 2026-06-15 10:37:39 -07:00
parent f8e0079d3f
commit 00d3203d68
51 changed files with 153 additions and 9889 deletions

View file

@ -1159,13 +1159,13 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
}
/**
* Increment count for entity type - O(1) operation
* Protected by storage-specific mechanisms (mutex, distributed consensus, etc.)
* Increment count for entity type - O(1) operation.
* Concurrency is handled by the process-global mutex
* ({@link incrementEntityCountSafe}); this raw form is for callers that
* already hold it.
* @param type The entity type
*/
protected incrementEntityCount(type: string): void {
// For distributed scenarios, this is aggregated across shards
// For single-node, this is protected by storage-specific locking
this.entityCounts.set(type, (this.entityCounts.get(type) || 0) + 1)
this.totalNounCount++
// Update cache
@ -1176,11 +1176,11 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
}
/**
* Thread-safe increment for concurrent scenarios
* Uses mutex for single-node, distributed consensus for multi-node
* Thread-safe increment for concurrent scenarios.
* A process-global mutex serialises the read-modify-write so concurrent
* writers in the same process cannot lose updates.
*/
protected async incrementEntityCountSafe(type: string): Promise<void> {
// Single-node mutex protection (distributed mode handled by coordinator)
const mutex = getGlobalMutex()
await mutex.runExclusive(`count-entity-${type}`, async () => {
this.incrementEntityCount(type)
@ -1225,8 +1225,10 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
}
/**
* Increment verb count - O(1) operation (now synchronous)
* Protected by storage-specific mechanisms (mutex, distributed consensus, etc.)
* Increment verb count - O(1) operation (now synchronous).
* Concurrency is handled by the process-global mutex
* ({@link incrementVerbCountSafe}); this raw form is for callers that already
* hold it.
* @param type The verb type
*/
protected incrementVerbCount(type: string): void {
@ -1240,8 +1242,9 @@ export abstract class BaseStorageAdapter implements StorageAdapter {
}
/**
* Thread-safe increment for verb counts
* Uses mutex for single-node, distributed consensus for multi-node
* Thread-safe increment for verb counts.
* A process-global mutex serialises the read-modify-write so concurrent
* writers in the same process cannot lose updates.
* @param type The verb type
*/
protected async incrementVerbCountSafe(type: string): Promise<void> {