feat(8.0): brain.fillSubtypes migration helper + pre-RC1 gap closure
- brain.fillSubtypes(rules): idempotent subtype back-fill for pre-8.0 data.
One rule per NounType/VerbType (literal default or per-entry function);
fills only entries still missing a subtype through the public update()/
updateRelation() paths; returns { scanned, filled, skipped, errors, byType }.
Full unit suite in tests/unit/brainy/fill-subtypes.test.ts.
- Fix getNouns/getVerbs pagination hasMore (peek one past the window) —
was permanently false, silently truncating every multi-page walk.
- find({ near }) without near.id now throws a teaching error instead of an
opaque storage sharding failure; CLI --threshold without --near applies a
plain score floor.
- CLI init/close audit: every one-shot command init()s, close()s, and exits
explicitly; delete the unmaintained interactive REPL; replace the cloud-era
storage subcommands with status/batch-delete; new types/validate commands.
- requireSubtype JSDoc now documents the 8.0 default-on contract; audit()
recommendation points at fillSubtypes.
- Docs: data-storage-architecture rewritten to the real 8.0 on-disk layout;
README storage section reflects filesystem+memory and snapshots; eli5 and
SEMANTIC_VFS /as-of/ semantics corrected; internal tracker IDs and
.strategy references scrubbed from published files.
This commit is contained in:
parent
9b0f4acd5b
commit
c44678390e
30 changed files with 1517 additions and 3226 deletions
|
|
@ -748,9 +748,8 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
*
|
||||
* If neither the column store nor a sparse index covers the field, the
|
||||
* function throws `BrainyError(FIELD_NOT_INDEXED)`. Returning `[]` for a
|
||||
* genuinely unindexed field was the bug class `BR-FIND-WHERE-ZERO`
|
||||
* tracked — a silent empty result indistinguishable from "the data
|
||||
* really isn't there."
|
||||
* genuinely unindexed field was a long-standing silent-empty bug class —
|
||||
* an empty result indistinguishable from "the data really isn't there."
|
||||
*/
|
||||
private async getIdsFromChunks(field: string, value: any): Promise<string[]> {
|
||||
// Load sparse index via UnifiedCache (lazy loading)
|
||||
|
|
@ -1200,7 +1199,8 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
// - Chunked Sparse Index: ~50 values per chunk, lazy-loaded
|
||||
// - UnifiedCache LRU: Only hot chunks in memory
|
||||
//
|
||||
// Future: Bloom filter hybrid for unlimited words (see .strategy/BILLION-SCALE-PLAN.md)
|
||||
// A Bloom-filter hybrid could lift the per-entity word cap entirely if
|
||||
// full-document indexing at billion-entity scale ever becomes a need.
|
||||
const textContent = this.extractTextContent(data)
|
||||
if (textContent) {
|
||||
const MAX_WORDS_PER_ENTITY = 5000 // Handles articles/chapters, memory-safe at scale
|
||||
|
|
@ -1600,8 +1600,8 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
* Throws `BrainyError(FIELD_NOT_INDEXED)` if the field has no entries in
|
||||
* either store. Callers in find()-evaluation catch this and translate to
|
||||
* an empty result with a logged warning. The throw aligns the production
|
||||
* `find()` path with the `brain.explain()` diagnostic, so the silent-
|
||||
* empty bug class (BR-FIND-WHERE-ZERO) is no longer possible.
|
||||
* `find()` path with the `brain.explain()` diagnostic, so a silently
|
||||
* empty result for an unindexed field is no longer possible.
|
||||
*/
|
||||
async getIds(field: string, value: any): Promise<string[]> {
|
||||
// Track exact query for field statistics
|
||||
|
|
@ -2866,8 +2866,8 @@ export class MetadataIndexManager implements MetadataIndexProvider {
|
|||
*
|
||||
* Prior implementation read from `this.fieldIndexes` + lazy-loaded sparse
|
||||
* indices, which silently returned `0` entries for any workspace written
|
||||
* after sparse-index writes were deleted in commit `11be039`. That defect
|
||||
* is what `BR-FIND-WHERE-ZERO` tracked.
|
||||
* after sparse-index writes were deleted in commit `11be039`. That
|
||||
* silent-zero defect is why this reads the column store first.
|
||||
*/
|
||||
async getStats(): Promise<MetadataIndexStats> {
|
||||
const entityCount = this.idMapper.size
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue