Commit graph

8 commits

Author SHA1 Message Date
40d2cd5419 docs(8.0): correct public docs to the real 8.0 API + honest perf claims
The GA-readiness audit found the public docs had drifted from the shipped
surface and presented uncited performance numbers as measured fact.

- quick-start: `FindResult`→`Result`, `VerbType.BuiltOn`→`DependsOn` (the
  canonical getting-started example now compiles).
- noun-verb-taxonomy: rewrote every sample off removed/fictional APIs
  (`augment`/`connectModel`/`getVerbs`/two-arg `add`/`like`/`$gte`) onto the
  real single-object `add`/`find`/`relate`/`related`; replaced the stale
  31-noun/40-verb catalogs with accurate, complete tables (42 nouns, 127 verbs).
- triple-intelligence: `like:`→`query:`, dollar-operators→bare operators, and
  several other fictional keys swept to the real `FindParams`.
- FIND_SYSTEM / PERFORMANCE / index-architecture / BATCHING: replaced
  fabricated, mutually-inconsistent latency tables and uncited speedup
  multipliers with Big-O characterizations, qualitative mechanism descriptions,
  and the one genuinely-measured benchmark (graph O(1) neighbor lookup), per the
  evidence-based-claims rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 10:03:02 -07:00
1f7e365a4e chore(8.0): final pre-RC1 sweep — API consistency, named errors, orphans, zero-cast codebase 2026-06-11 14:51:00 -07:00
cc8037db10 docs(8.0): consistency-model concept + snapshots guide — Db API replaces branching docs 2026-06-11 08:37:26 -07:00
adda1570f3 docs(8.0): Phase F — deep clean across 21 docs
Aligned every public doc to the 8.0 contract: filesystem + memory adapters
only, vector index provider terminology (config.vector with recall +
quantization + persistMode knobs), no cloud storage adapters, no closed-
source product names.

Tier 1 — heavier rewrites:
- docs/architecture/storage-architecture.md
- docs/architecture/data-storage-architecture.md
- docs/architecture/distributed-storage.md DELETED — content was 100%
  cloud-coordination examples with no 8.0 substance.
- docs/guides/distributed-system.md DELETED — same reason; no inbound refs.
- docs/SCALING.md rewritten for single-node guidance.
- docs/PLUGINS.md, docs/augmentations/{COMPLETE-REFERENCE,README}.md:
  HnswProvider→VectorIndexProvider, hnsw→vector key.
- docs/PERFORMANCE.md, docs/BATCHING.md cloud-detection + sharding
  sections replaced with single-node vector tuning + filesystem framing.

Tier 2 — surgical renames + cloud-section deletions:
- architecture/{index,initialization-and-rebuild,overview}.md
- transactions.md, DEVELOPER_LEARNING_PATH.md
- vfs/{VFS_API_GUIDE,COMMON_PATTERNS}.md
- api/README.md, guides/{inspection,import-flow}.md

Tier 3 — light edits:
- docs/README.md, architecture/augmentation-system-audit.md

MIGRATION-V3-TO-V4.md untouched (internal migration doc, no stale terms).
2026-06-09 16:13:35 -07:00
b6e3470b83 docs: add public frontmatter to docs for soulcraft.com/docs pipeline
Add YAML frontmatter (slug, public, category, template, order) to 8
existing docs and 2 new getting-started guides (installation, quick-start).
Include docs/**/*.md in npm package files so the portal sync-docs script
can read them from node_modules after publish.

Update CLAUDE.md with docs pipeline trigger phrases and release checklist.
2026-02-19 17:04:05 -08:00
364360d447 fix: exclude __words__ keyword index from corruption detection and getStats()
The __words__ keyword index stores 50-5000 entries per entity (one per
word), which inflated avg entries/entity well above the corruption
threshold of 100. This caused:

1. validateConsistency() to falsely detect corruption on every startup,
   triggering unnecessary clearAllIndexData() + rebuild() cycles
2. getStats() to log false "Metadata index may be corrupted" warnings
   and report inflated totalEntries/totalIds stats

Both methods now skip __words__ when counting, so stats and health
checks reflect metadata fields only (noun, type, createdAt, etc.).
Keyword search is unaffected since the __words__ field index itself
is not modified.
2026-01-27 15:38:21 -08:00
42ae5be455 feat: ID-first storage architecture + remove memory-unsafe APIs (v6.0.0)
BREAKING CHANGES:

**ID-First Storage Paths**
- Direct O(1) entity access without type lookups
- Before: entities/nouns/{TYPE}/metadata/{SHARD}/{ID}.json
- After:  entities/nouns/{SHARD}/{ID}/metadata.json
- Migration handled automatically on first init()

**Removed Memory-Unsafe APIs**
- Removed brain.merge() - loaded all entities into memory
- Removed brain.diff() - loaded all entities into memory
- Removed brain.data().backup() - loaded all entities into memory
- Removed brain.data().restore() - depended on backup()
- Removed CLI commands: backup, restore, cow merge

**Migration Paths**
- merge() → Use checkout() or manually copy entities with pagination
- diff() → Use asOf() with manual paginated comparison
- backup() → Use fork() for instant COW snapshots
- restore() → Use checkout() to switch to snapshot branch

Core Improvements:
-  All 8 storage adapters properly call super.init()
-  GraphAdjacencyIndex integration in BaseStorage.init()
-  Fixed ID-first path bugs (vector.json → vectors.json)
-  Fixed MemoryStorage.initializeCounts() for ID-first paths
-  New VFS APIs: du(), access(), find()
-  Comprehensive documentation with migration guides

Storage Adapters Fixed:
- MemoryStorage, FileSystemStorage, AzureBlobStorage
- GCSStorage, R2Storage, S3CompatibleStorage
- OPFSStorage, HistoricalStorageAdapter

Files Changed: 28 files, +1,075/-1,933 lines (net -858)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 16:46:11 -08:00
95cbab2e3f feat: add storage-level batch operations to eliminate N+1 query patterns
Implements comprehensive batching infrastructure (brain.batchGet, storage.getNounMetadataBatch, storage.getVerbsBySourceBatch) with native cloud adapter APIs for GCS, S3, R2, and Azure. VFS operations now use parallel breadth-first traversal with batching, reducing directory reads from 22 sequential calls to 2-3 batched calls. Improves cloud storage performance by 90%+ (12.7s → <1s for 12 files). Fully compatible with type-aware storage, sharding, COW, fork(), and all indexes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 08:59:11 -08:00