docs(8.0): Phase F — deep clean across 21 docs
Aligned every public doc to the 8.0 contract: filesystem + memory adapters
only, vector index provider terminology (config.vector with recall +
quantization + persistMode knobs), no cloud storage adapters, no closed-
source product names.
Tier 1 — heavier rewrites:
- docs/architecture/storage-architecture.md
- docs/architecture/data-storage-architecture.md
- docs/architecture/distributed-storage.md DELETED — content was 100%
cloud-coordination examples with no 8.0 substance.
- docs/guides/distributed-system.md DELETED — same reason; no inbound refs.
- docs/SCALING.md rewritten for single-node guidance.
- docs/PLUGINS.md, docs/augmentations/{COMPLETE-REFERENCE,README}.md:
HnswProvider→VectorIndexProvider, hnsw→vector key.
- docs/PERFORMANCE.md, docs/BATCHING.md cloud-detection + sharding
sections replaced with single-node vector tuning + filesystem framing.
Tier 2 — surgical renames + cloud-section deletions:
- architecture/{index,initialization-and-rebuild,overview}.md
- transactions.md, DEVELOPER_LEARNING_PATH.md
- vfs/{VFS_API_GUIDE,COMMON_PATTERNS}.md
- api/README.md, guides/{inspection,import-flow}.md
Tier 3 — light edits:
- docs/README.md, architecture/augmentation-system-audit.md
MIGRATION-V3-TO-V4.md untouched (internal migration doc, no stale terms).
This commit is contained in:
parent
2626ab8d62
commit
adda1570f3
22 changed files with 570 additions and 2658 deletions
|
|
@ -65,7 +65,7 @@ Semantic vectors with metadata and relationships - the fundamental data unit in
|
|||
Typed connections between entities with optional `data` and `metadata` - building knowledge graphs.
|
||||
|
||||
### 📊 Data vs Metadata
|
||||
- **`data`**: Content embedded into vectors. Searchable via **semantic similarity** (HNSW) and **hybrid text+semantic** search. NOT queryable via `where` filters.
|
||||
- **`data`**: Content embedded into vectors. Searchable via **semantic similarity** (vector index) and **hybrid text+semantic** search. NOT queryable via `where` filters.
|
||||
- **`metadata`**: Structured fields indexed by MetadataIndex. Queryable via `where` filters in `find()`.
|
||||
|
||||
See **[Data Model](../DATA_MODEL.md)** for the full explanation.
|
||||
|
|
@ -232,7 +232,7 @@ const results = await brain.find({
|
|||
- **Advanced:** Object with vector + graph + metadata filters
|
||||
|
||||
**FindParams:**
|
||||
- `query?`: `string` - Text for semantic + hybrid search (searches `data` via HNSW + text index)
|
||||
- `query?`: `string` - Text for semantic + hybrid search (searches `data` via the vector index + text index)
|
||||
- `type?`: `NounType | NounType[]` - Filter by entity type(s). Alias for `where.noun`.
|
||||
- `subtype?`: `string | string[]` - Filter by sub-classification (top-level standard field, fast path). Single string for equality, array for set membership.
|
||||
- `where?`: `object` - Metadata filters. See **[Query Operators](../QUERY_OPERATORS.md)** for all operators.
|
||||
|
|
@ -808,7 +808,7 @@ const parentData = await brain.find({}) // Original data unchanged
|
|||
|
||||
**Returns:** `Promise<Brainy>` - New Brainy instance on forked branch
|
||||
|
||||
**How it works:** Snowflake-style COW shares HNSW index, copies only modified nodes (10-20% memory overhead).
|
||||
**How it works:** Snowflake-style COW shares the vector index, copies only modified nodes (10-20% memory overhead).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -1787,28 +1787,20 @@ const entities = await snapshot.find({ limit: 100 })
|
|||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
// Storage configuration
|
||||
storage: {
|
||||
type: 'memory', // memory | opfs | filesystem | s3 | r2 | gcs | azure
|
||||
path: './brainy-data', // For filesystem storage
|
||||
compression: true, // Enable gzip compression (60-80% savings)
|
||||
// Storage configuration
|
||||
storage: {
|
||||
type: 'filesystem', // 'memory' | 'filesystem' | 'auto'
|
||||
rootDirectory: './brainy-data'
|
||||
},
|
||||
|
||||
// Cloud storage configs (see Storage Adapters section)
|
||||
s3Storage: { ... },
|
||||
r2Storage: { ... },
|
||||
gcsStorage: { ... },
|
||||
azureStorage: { ... }
|
||||
},
|
||||
// Vector index configuration (3 knobs)
|
||||
vector: {
|
||||
recall: 'balanced', // 'fast' | 'balanced' | 'accurate'
|
||||
quantization: { bits: 8 }, // 4 | 8 (SQ4 / SQ8)
|
||||
persistMode: 'immediate' // 'immediate' | 'deferred'
|
||||
},
|
||||
|
||||
// HNSW vector index config
|
||||
hnsw: {
|
||||
M: 16, // Connections per layer
|
||||
efConstruction: 200, // Construction quality
|
||||
efSearch: 100, // Search quality
|
||||
typeAware: true // Enable type-aware indexing
|
||||
},
|
||||
|
||||
// Model configuration (embedded in WASM - zero config needed)
|
||||
// Model configuration (embedded in WASM - zero config needed)
|
||||
// Model: all-MiniLM-L6-v2 (384 dimensions)
|
||||
// Device: CPU via WASM (works everywhere)
|
||||
|
||||
|
|
@ -1827,13 +1819,13 @@ await brain.init() // Required! VFS auto-initialized
|
|||
|
||||
## Storage Adapters
|
||||
|
||||
All 7 storage adapters support **copy-on-write branching**.
|
||||
Brainy 8.0 ships two adapters — both support **copy-on-write branching**.
|
||||
|
||||
### Memory (Default)
|
||||
### Memory (Default for Tests)
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'memory' }
|
||||
storage: { type: 'memory' }
|
||||
})
|
||||
```
|
||||
|
||||
|
|
@ -1841,125 +1833,32 @@ const brain = new Brainy({
|
|||
|
||||
---
|
||||
|
||||
### OPFS (Browser)
|
||||
### Filesystem (Default for Node)
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: { type: 'opfs' }
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
rootDirectory: './brainy-data'
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Use case:** Browser applications with persistent storage
|
||||
**Use case:** Node.js applications, single-node production deployments
|
||||
|
||||
For off-site backup, snapshot `rootDirectory` from your scheduler (`gsutil rsync`, `aws s3 sync`, `rclone`, or `tar`) — Brainy itself doesn't reach out to cloud object stores.
|
||||
|
||||
---
|
||||
|
||||
### Filesystem (Node.js)
|
||||
### Auto
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'filesystem',
|
||||
path: './brainy-data',
|
||||
compression: true // 60-80% space savings
|
||||
}
|
||||
storage: { type: 'auto', rootDirectory: './brainy-data' }
|
||||
})
|
||||
```
|
||||
|
||||
**Use case:** Node.js applications, local persistence
|
||||
|
||||
---
|
||||
|
||||
### AWS S3
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 's3',
|
||||
s3Storage: {
|
||||
bucketName: 'my-brainy-data',
|
||||
region: 'us-east-1',
|
||||
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
|
||||
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
// Enable Intelligent-Tiering for 96% cost savings
|
||||
await brain.storage.enableIntelligentTiering('entities/', 'auto-tier')
|
||||
```
|
||||
|
||||
**Use case:** Production deployments, scalable storage
|
||||
|
||||
**[📖 AWS S3 Cost Optimization →](../operations/cost-optimization-aws-s3.md)**
|
||||
|
||||
---
|
||||
|
||||
### Cloudflare R2
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'r2',
|
||||
r2Storage: {
|
||||
accountId: process.env.CF_ACCOUNT_ID,
|
||||
bucketName: 'my-brainy-data',
|
||||
accessKeyId: process.env.CF_ACCESS_KEY_ID,
|
||||
secretAccessKey: process.env.CF_SECRET_ACCESS_KEY
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Use case:** Zero egress fees, cost-effective storage
|
||||
|
||||
**[📖 R2 Cost Optimization →](../operations/cost-optimization-cloudflare-r2.md)**
|
||||
|
||||
---
|
||||
|
||||
### Google Cloud Storage (GCS)
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
gcsStorage: {
|
||||
bucketName: 'my-brainy-data',
|
||||
projectId: process.env.GCP_PROJECT_ID,
|
||||
keyFilename: './gcp-key.json'
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
// Enable auto-tiering
|
||||
await brain.storage.enableAutoclass({
|
||||
terminalStorageClass: 'ARCHIVE'
|
||||
})
|
||||
```
|
||||
|
||||
**Use case:** Google Cloud ecosystem, global distribution
|
||||
|
||||
**[📖 GCS Cost Optimization →](../operations/cost-optimization-gcs.md)**
|
||||
|
||||
---
|
||||
|
||||
### Azure Blob Storage
|
||||
|
||||
```typescript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'azure',
|
||||
azureStorage: {
|
||||
accountName: process.env.AZURE_STORAGE_ACCOUNT,
|
||||
accountKey: process.env.AZURE_STORAGE_KEY,
|
||||
containerName: 'brainy-data'
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Use case:** Azure ecosystem, enterprise deployments
|
||||
|
||||
**[📖 Azure Cost Optimization →](../operations/cost-optimization-azure.md)**
|
||||
Picks `'filesystem'` on Node with a writable `rootDirectory`, falls back to `'memory'` otherwise.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -2292,7 +2191,7 @@ console.log(`Fields: ${stats.metadataFields.join(', ')}`)
|
|||
|
||||
**Returns:**
|
||||
- `entities` - Total entity count
|
||||
- `vectors` - Total vectors in HNSW index
|
||||
- `vectors` - Total vectors in the vector index
|
||||
- `relationships` - Total relationships in graph
|
||||
- `metadataFields` - Indexed metadata fields
|
||||
- `memoryUsage.vectors` - Vector memory (bytes)
|
||||
|
|
@ -2597,7 +2496,7 @@ For the full taxonomy with all 169 types and their descriptions, see:
|
|||
- ✅ **Git-Style Branching** - fork, commit, checkout, listBranches
|
||||
- ✅ **Full Branch Isolation** - Parent and fork fully isolated
|
||||
- ✅ **Read-Through Inheritance** - Forks see parent + own data
|
||||
- ✅ **Universal Storage Support** - All 7 adapters support branching
|
||||
- ✅ **Universal Storage Support** - Filesystem and memory adapters both support branching
|
||||
|
||||
**[📖 Complete Changes →](../../.strategy/v5.1.0-CHANGES.md)**
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue