feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure
- brain.fillSubtypes(rules): idempotent subtype back-fill for pre-8.0 data.
One rule per NounType/VerbType (literal default or per-entry function);
fills only entries still missing a subtype through the public update()/
updateRelation() paths; returns { scanned, filled, skipped, errors, byType }.
Full unit suite in tests/unit/brainy/fill-subtypes.test.ts.
- Fix getNouns/getVerbs pagination hasMore (peek one past the window) —
was permanently false, silently truncating every multi-page walk.
- find({ near }) without near.id now throws a teaching error instead of an
opaque storage sharding failure; CLI --threshold without --near applies a
plain score floor.
- CLI init/close audit: every one-shot command init()s, close()s, and exits
explicitly; delete the unmaintained interactive REPL; replace the cloud-era
storage subcommands with status/batch-delete; new types/validate commands.
- requireSubtype JSDoc now documents the 8.0 default-on contract; audit()
recommendation points at fillSubtypes.
- Docs: data-storage-architecture rewritten to the real 8.0 on-disk layout;
README storage section reflects filesystem+memory and snapshots; eli5 and
SEMANTIC_VFS /as-of/ semantics corrected; internal tracker IDs and
.strategy references scrubbed from published files.
This commit is contained in:
parent
9b0f4acd5b
commit
c44678390e
30 changed files with 1517 additions and 3226 deletions
|
|
@ -47,16 +47,18 @@ const authFiles = await vfs.readdir('/by-concept/authentication')
|
|||
const aliceFiles = await vfs.readdir('/by-author/alice')
|
||||
```
|
||||
|
||||
### 2. **Time Travel**
|
||||
See your codebase as it existed at any point:
|
||||
### 2. **Change Tracking by Date**
|
||||
List the files that were modified on any given day:
|
||||
```typescript
|
||||
// Code from March 15th
|
||||
const snapshot = await vfs.readdir('/as-of/2024-03-15')
|
||||
// Files that changed on March 15th
|
||||
const changed = await vfs.readdir('/as-of/2024-03-15')
|
||||
|
||||
// Compare with today
|
||||
// Everything under /src right now
|
||||
const current = await vfs.readdir('/src')
|
||||
```
|
||||
|
||||
`/as-of/<date>` selects by *modification date* — it reads the files' current content, not historical versions. For true point-in-time queries over entity state, use the Db API (`brain.asOf(generation)`).
|
||||
|
||||
### 3. **Knowledge Graph Navigation**
|
||||
Navigate by semantic relationships:
|
||||
```typescript
|
||||
|
|
@ -118,13 +120,14 @@ await vfs.stat('/by-author/alice/config.ts')
|
|||
### 4. By Time (Temporal) ✅ **Production**
|
||||
```typescript
|
||||
await vfs.readdir('/as-of/2024-03-15')
|
||||
// Files modified on March 15, 2024
|
||||
// Files modified on March 15, 2024 (24-hour window)
|
||||
|
||||
await vfs.readFile('/as-of/2024-03-15/src/auth.ts')
|
||||
// Read auth.ts as it existed that day
|
||||
await vfs.readFile('/as-of/2024-03-15/auth.ts')
|
||||
// Current content of auth.ts, addressed by modification date —
|
||||
// the path only resolves if auth.ts was modified that day
|
||||
```
|
||||
|
||||
**How it works:** Tracks `modified` timestamp. Uses B-tree range queries (`greaterEqual`/`lessEqual`) for O(log n) performance.
|
||||
**How it works:** Tracks the `modified` timestamp on every file and runs a range query (`greaterEqual`/`lessEqual`) over one 24-hour window for O(log n) performance. The VFS does not store historical file contents — `/as-of/` filters by *when a file last changed*; reads return the current bytes. For point-in-time state, use the Db API (`brain.asOf(generation)`).
|
||||
|
||||
**Status:** ✅ Fully implemented and tested at 10K file scale
|
||||
|
||||
|
|
@ -222,7 +225,7 @@ console.log(authFiles)
|
|||
// ['login.ts', 'signup.ts', 'oauth.ts']
|
||||
```
|
||||
|
||||
### Example 2: Time Travel
|
||||
### Example 2: Changes by Day
|
||||
```typescript
|
||||
// See what changed today
|
||||
const today = new Date().toISOString().split('T')[0]
|
||||
|
|
@ -232,8 +235,8 @@ const todaysFiles = await vfs.readdir(`/as-of/${today}`)
|
|||
const yesterday = new Date(Date.now() - 86400000).toISOString().split('T')[0]
|
||||
const yesterdaysFiles = await vfs.readdir(`/as-of/${yesterday}`)
|
||||
|
||||
const newFiles = todaysFiles.filter(f => !yesterdaysFiles.includes(f))
|
||||
console.log('New files today:', newFiles)
|
||||
const onlyToday = todaysFiles.filter(f => !yesterdaysFiles.includes(f))
|
||||
console.log('Changed today (untouched yesterday):', onlyToday)
|
||||
```
|
||||
|
||||
### Example 3: Graph Navigation
|
||||
|
|
@ -432,25 +435,17 @@ await vfs.writeFile(path, code, {
|
|||
})
|
||||
```
|
||||
|
||||
### 3. Optimize for Your Scale
|
||||
### 3. Combine Dimensions
|
||||
```typescript
|
||||
// For < 100K files: Post-filtering is fine
|
||||
// For > 100K files: Use flattened indexes
|
||||
|
||||
// Force index refresh after bulk operations
|
||||
await brain.storage.rebuildIndexes()
|
||||
```
|
||||
|
||||
### 4. Combine Dimensions
|
||||
```typescript
|
||||
// Find security files Alice worked on this week
|
||||
// Find security files Alice changed on a given day
|
||||
// (each /as-of/<date> path covers exactly that one day)
|
||||
const aliceFiles = await vfs.readdir('/by-author/alice')
|
||||
const securityFiles = await vfs.readdir('/by-tag/security')
|
||||
const thisWeek = await vfs.readdir(`/as-of/${weekAgo}`)
|
||||
const changedThatDay = await vfs.readdir('/as-of/2024-03-15')
|
||||
|
||||
const intersection = aliceFiles
|
||||
.filter(f => securityFiles.includes(f))
|
||||
.filter(f => thisWeek.includes(f))
|
||||
.filter(f => changedThatDay.includes(f))
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -469,14 +464,14 @@ console.log(entity.metadata.concepts)
|
|||
|
||||
### Slow Queries on Large Datasets
|
||||
```typescript
|
||||
// Check if indexes are built
|
||||
const stats = await brain.storage.getIndexStats()
|
||||
// Check if indexes are built and populated
|
||||
const stats = await brain.getIndexStats()
|
||||
console.log(stats)
|
||||
|
||||
// Rebuild if needed
|
||||
await brain.storage.rebuildIndexes()
|
||||
```
|
||||
|
||||
If an index looks empty or inconsistent, rebuild from raw storage with the CLI
|
||||
(stop the live writer first): `brainy inspect repair <data-dir>`.
|
||||
|
||||
### Semantic Path Returns Empty
|
||||
```typescript
|
||||
// Check if metadata exists
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue