Step 11 POSTed to repos/soulcraft/brainy while printing the correct URL; dormant only because FORGEJO_RELEASE_TOKEN was unset. Found during the 10.4.4 cut verification.
The manifest's prose pointer named a document that answers a confidential
specification, and such a document does not belong in a public repository even
in summary. The pointer is dropped — the manifest is generated from this
engine's own surface and is self-describing — and the requirement marking it
deliberately omits is recorded with the contract's owner rather than here.
The standard is written down so this is not relitigated per document.
Open Brainy's side of the API contract the accelerated engine published.
DECLARED: package.json carries "brainyContract": 1 and the engine states its
own via contractVersion() / BRAINY_CONTRACT_VERSION — two engines compare an
integer instead of probing prototypes, and a tool reads the package field
without importing the engine. Pinned so the two can never drift apart.
SERVED: hasAll, noneOf and excludes now work on the index path. The defect
underneath was worse than the reported divergence — the metadata index's
operator switch had NO DEFAULT CASE, so any operator without a case left the
field's match set at its initial [] and find() returned an empty page.
Documented, validator-accepted, matcher-implemented operators answering
silently wrong. hasAll intersects each element's posting set (an empty operand
is vacuously true of every row that has the field), noneOf complements their
union, excludes complements contains.
REFUSED BY NAME: startsWith, endsWith, matches and length raise
INVALID_QUERY naming the operator, the field and the reason. An equality/range
posting index cannot evaluate a substring, a pattern or an array length without
reading every row — which is the cost this path exists to avoid — so it refuses
rather than answering an empty page. Both engines now agree on all 25 tokens
and contract 1 has no remaining operator divergence. This is a visible change
for a consumer calling those four through find({ where }): an empty page
becomes a typed refusal.
EMITTED: scripts/emit-contract-manifest.mjs generates docs/api-contract.json
from the BUILT surface — prototype doors, exported error classes, the operator
sets read out of their single definitions, the field-addressing vocabulary, the
health verdicts. Nothing hand-maintained, so a diff between two manifests is a
diff between two engines. `--check` fails on a stale manifest, which makes the
announce-every-addition duty mechanical rather than remembered.
RATIFIED in docs/contract-1-ratification.md: the 41-of-57 required split with
the promise spelled out (a refusal is part of a door; deprecation is not
removal), the serving-withholding list confirmed exhaustive and identical, the
minor/major rule adopted with the announcement duty, the 30 storage seam
methods committed as supported surface until Stage 2, and a finding filed
against the spec — is / isNot / greaterEqual / lessEqual are listed there as
served aliases and have never existed in this engine, which throws
INVALID_QUERY on all four.
Prepares the repo for its new home at soulcraftlabs/open-brainy ahead
of the Forgejo transfer: package name, publish registry, release
script, and every install/import reference across docs, src, tests,
examples, and integrations now point at @soulcraftlabs/brainy on
The Source. The npmjs storefront leg and byte-identity pair
verification are stripped from the release script — The Source is
now the only publish target. README gains an Open Brainy explainer
and a registry note for consumers.
@soulcraft/brainy 10.4.2 was the last release under the old name.
Two builds of the same source tree could publish different artifacts
because buildEmbeddedPatterns.ts and buildTypeEmbeddings.ts stamped
their generated output with new Date().toISOString(). Route both
generators' "Generated:" header and the generatedAt runtime field
through a shared resolver: newest git commit time among the
generator's inputs (script + source data), falling back to the stamp
already present in the previous output when git is unavailable (e.g.
a published tarball build), and finally to a fixed epoch value. Every
fallback logs to stderr so degradation is never silent.
Regenerated both committed output files once so the tree carries
deterministic stamps; no other content changed.
A prerelease the other engine devDeps from our own registry while the two
are being proven together must never reach npmjs: --source-only tags, lets
CI publish to The Source, creates the release page, and SKIPS the storefront
publish, the cross-registry verification and the docs push — loudly, at
each step. Refused for a non-prerelease version: a public floor always ships
byte-identical on both registries.
Two guards for every gate lane, born from the 2026-08-13 lost-day ledger.
gate-preflight.sh refuses a lane on a machine that cannot be trusted to
produce honest numbers — co-tenant processes named by pid and command, load
average, CPU governor, disk floors — one FATAL line per violation so the
operator can act from the message alone. vitest-verdict-check.sh refuses a
suite log that cannot be trusted as a verdict — missing or mismatched
summary counts, files that never executed (a truncated run once read as
green from three files of ninety-nine), and worker-pool death signatures.
Both verified live: the preflight correctly refuses this workstation naming
its actual offenders; the verdict guard passes/fails five fixture shapes
(clean, wrong-count, truncated, worker-death, no-summary) and both CLI
modes. Wire-up into the CI lanes rides the runner program.
The fold checkpoint (_system/fold-checkpoint.json) is stamped strictly after
a canonical-sync barrier over every live entity touched since the last stamp
(syncEntityCanonical: ids → canonical paths → fsync; an absent file fsyncs
its parent directory so deletes are as durable as writes). An unclean open
under log authority now folds only (checkpoint, head]; the chain bootstraps
at an empty brain's adoption (three-phase hooks around adoptLogAuthority) or
at a brain's first whole-log fold — existing brains converge at their first
crash with zero regression. Rollback restores sync immediately; abort paths
feed the barrier; a failed barrier retains the old bound (bigger fold later,
never a lost write). Five structural pins including boundedness itself.
Also: the production-shaped write-flow gate leg (mixed traffic racing
flushes, crash mid-traffic, every ack survives — from a consumer-reported
gate miss), and two release-ceremony cures (tag-first push so the publish
never queues behind the release commit's CI run; raw-curl npmjs shasum
probe with propagation grace instead of a one-shot false divergence).
Ruled today: the project's one public home is source.soulcraft.com. The
old public repo is archived history and no longer part of any release.
- package.json repository/homepage/bugs now point at the forge (this is
what the npm page links as Repository/Homepage/Issues)
- README CI badge reads the forge pipeline; CONTRIBUTING drops the
mirror paragraph (forge account or email patch were already the ruled
contribution paths)
- release.sh: mirror push + external release step removed; publishes go
forge-first (box-held write token, temp userconfig so the token never
hits argv; a forge-publish failure aborts before the storefront so the
pair can never diverge), then npmjs with the scope-override pin (the
fleet npmrc maps @soulcraft to the forge and scope mappings beat
--registry); release page created via forge API when a token is
present, loud skip otherwise; changelog compare links point home
- dead external CI workflow removed (.forgejo/workflows/ci.yml is the
live pipeline)
Historical CHANGELOG links to the archive stay as written - history is
history and the archive serves them read-only.
Origin moved to the private forge; the public repo is now a mirror with an
unknown sync cadence. The release flow creates the GitHub release directly,
and a missing tag there would be silently created at the default-branch
head - the wrong commit. The script now pushes branch+tag to the mirror
itself and hard-stops if the tag's commit on the mirror differs from local,
before anything irreversible (npm publish) happens.
scripts/push-docs.js collects docs/**/*.md with public:true frontmatter
and POSTs them (batches of 10, idempotent per slug) to the docs ingest
door after npm publish — release.sh step 12. Absent secret = loud skip
(publish already happened; the serving side can interim-sync); a failed
push exits non-zero so the docs site never silently trails npm. The
combined /docs landing index is deliberately NOT pushed per-repo — it
spans both engine corpora and is authored on the serving side.
- id-normalization (#18): `brain.newId()` (UUID v7) + v7 default ids; non-UUID string ids are
transparently normalized to a stable UUID v5 on creation AND every lookup — get/update/remove,
relate(from,to), related, find({connected}), getMany, removeMany, the transact op-handler, and
db.with() overlays — with the caller's original key preserved under `_originalId`. The engine only
ever sees a UUID; real UUIDs pass through untouched. universal/uuid.ts gains v5/v7/isUUID +
BRAINY_ID_NAMESPACE; new utils/idNormalization.ts (coerceNewEntityId / resolveEntityId).
- aggregation min/max delete-safety: `queryAggregate` no longer leaves a stale min/max after
deleting the current extreme — min/max now track a value multiset and recompute in-memory (no
entity scan; that scan was the prior delete-then-hang a consumer reported). Other metrics were
already exact across deletes. Regression test proves resolve-after-delete + correct new min/max.
- release.sh RC-safe: explicit version arg, prerelease detection (npm `--tag rc` + GitHub
`--prerelease`), full `test:ci` gate (unit + integration), current-branch push, npm-access
verification after publish.
- native-engine references point at `@soulcraft/cor` (8.0's partner) in user-facing strings/docs.
Unit 1461/0 · integration 599/0 · tsc clean.
A lightweight tag is skipped by 'git push --follow-tags', so v$VERSION stayed
local-only and 'gh release create' failed at the end of the release. Use an
annotated tag (-a) so it pushes with the release commit.
- Remove @huggingface/transformers dependency (539MB native binaries)
- Add direct ONNX Runtime Web embedding engine
- Bundle all-MiniLM-L6-v2-q8 model (24MB, no runtime downloads)
- Works with Node.js, Bun, and bun build --compile
- Air-gap compatible: fully self-contained, no internet required
New WASM embedding components:
- WASMEmbeddingEngine: Main integration class
- WordPieceTokenizer: Pure TypeScript tokenizer
- EmbeddingPostProcessor: Mean pooling + L2 normalization
- ONNXInferenceEngine: Direct ONNX Runtime Web wrapper
- AssetLoader: Model file loading
Tests added:
- 11 WASM embedding integration tests
- 8 Bun compatibility tests
New npm scripts:
- test:wasm - Run WASM embedding tests
- test:bun - Run tests with Bun
- test:bun:compile - Build and run compiled binary
Implements Phase 1.5 (Comprehensive MIME Type Detection) and adds built-in image processing support to IntelligentImportAugmentation.
**New Features:**
- ImageHandler: Extracts image metadata (dimensions, format, color space) using sharp
- EXIF extraction: Camera data, GPS, timestamps using exifr library
- Support for JPEG, PNG, WebP, GIF, TIFF, BMP, SVG, HEIC, AVIF formats
- MimeTypeDetector: Unified MIME type detection with magic byte support
- FormatDetector: Enhanced with image format detection via MIME + magic bytes
**Architecture Fixes:**
- Fixed brain.import() augmentation pipeline integration (src/brainy.ts:3140-3154)
- Added parameter spreading for ImportSource objects to enable augmentation access
- Fixed metadata propagation through ImportCoordinator to final results
- Added augmentation data check in ImportCoordinator.extract()
**Integration:**
- ImageHandler registered as built-in handler alongside CSV, Excel, PDF
- Images import as 'media' entities with 'image' subtype
- Full metadata preserved in knowledge graph entities
- Configuration options: enableImage, extractEXIF, imageDefaults
**Test Coverage:**
- 15 integration tests (image-import.test.ts) - 100% passing
- 27 unit tests (image-handler.test.ts) - 100% passing
- Format detection tests for all supported image types
- Error handling and resilience tests
**Breaking Changes:** None - backward compatible
Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
Major optimization - all type embeddings now built into package:
Build-time generation:
- Created scripts/buildTypeEmbeddings.ts to generate all type embeddings
- Generates embeddings for 31 NounTypes + 40 VerbTypes at build time
- Stores as base64-encoded binary data in embeddedTypeEmbeddings.ts
- Added check script to rebuild only when needed
Updated all consumers:
- NeuralEntityExtractor: loads pre-computed embeddings (instant)
- BrainyTypes: loads pre-computed embeddings (instant init)
- NaturalLanguageProcessor: loads pre-computed embeddings (instant init)
Build process:
- Added npm run build:types to generate embeddings
- Added npm run build:types:if-needed for conditional rebuild
- Integrated into main build pipeline
- Auto-rebuilds only when types or build script change
Benefits:
- Zero runtime cost - embeddings loaded instantly
- Survives all container restarts
- All 71 types always available (31 nouns + 40 verbs)
- ~100KB memory overhead for permanent performance gain
- Eliminates 5-10 second initialization delay
This completes the type embedding optimization started in v3.32.5
- Add scripts/release.sh for automated build → test → commit → push → publish → release
- Fix .versionrc.json types configuration (was causing 'types.find is not a function' error)
- Remove problematic precommit/postcommit hooks that ran full test suite during release
- Keep standard-version as fallback option (npm run release:standard-version:*)
- New workflow is faster, more reliable, and easier to debug
Usage:
npm run release # patch release (3.20.4 → 3.20.5)
npm run release:minor # minor release (3.20.4 → 3.21.0)
npm run release:major # major release (3.20.4 → 4.0.0)
npm run release:dry # dry-run mode (no changes)
- Replace Brainy with TransformerEmbedding for embedding patterns
- Update initialization logic to support TransformerEmbedding features like `localFilesOnly`
- Adjust embedding logic to use TransformerEmbedding's embed method
- Remove unused Brainy-related code (e.g., close method)
- Fix FileSystemStorage counting non-existent files in totalCount
- Add safety check in BaseStorage to prevent hasMore:true with empty items
- Ensure pagination terminates correctly even with corrupted storage state
- Add incremental sorted index updates during CRUD operations for consistent <5ms range queries
- Implement parallel search optimization with vector, metadata, and graph intelligence fusion
- Fix metadata-only query handling to properly return results without vector search
- Fix NLP recursive call issue by using embed() instead of add()
- Add cardinality tracking for smart index optimization
- Store entity data in metadata for proper retrieval
- Add comprehensive performance documentation
This improves query performance from O(n) to O(log n) for range queries
and ensures consistent fast performance without lazy loading delays.
- Implements automatic fallback chain: CDN → GitHub → Hugging Face
- Adds Soulcraft CDN as primary model source (models.soulcraft.com)
- GitHub release tar.gz extraction as reliable backup
- Zero configuration required - fully automatic
- Guarantees same model (all-MiniLM-L6-v2) across all sources
- 384-dimensional embeddings for data compatibility
- Local caching after first download
- Production-ready with multiple redundancy layers
BREAKING CHANGE: Removed tar-stream dependency, now uses native tar command
BREAKING CHANGE: Remove hard delete option from deleteVerb() for consistent API
- Add complete metadata namespace architecture with O(1) soft delete performance
- Implement periodic cleanup system for old soft-deleted items
- Add restore methods for both nouns and verbs
- Require metadata contracts for all augmentations
- Eliminate namespace collisions with clean separation (_brainy, _augmentations, _audit)
- Optimize index performance using flattened dot-notation for O(1) lookups
- Add comprehensive augmentation safety system with type-safe access control
- Maintain full backward compatibility for existing data
- Add enterprise-grade cleanup with configurable age thresholds and batch processing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Rename check-patterns.js to check-patterns.cjs for CommonJS compatibility
- Update package.json to reference .cjs file
- This fixes ES module errors when checking if patterns need rebuild
- Add embeddedPatterns.ts to version control (544KB file)
- Add check-patterns.js script to detect when rebuild is needed
- Modify build scripts to only rebuild patterns when changed
- This avoids memory-intensive pattern embedding on every npm publish
- Patterns rarely change, so this significantly speeds up releases
