brainy/src
David Snelling 178ff02045 feat: content-type-aware compression policy in COW BlobStorage (2.5.0 #32)
BlobStorage.write() in `auto` compression mode now consults the new
`BlobWriteOptions.mimeType` and skips zstd for MIME types known to be
already heavily compressed — JPEG, PNG, WebP, MP4, WebM, MP3, ZIP, PDF,
Office formats, etc. Gzip/zstd over these formats wastes CPU for no
measurable byte savings; the payload entropy is already near maximal,
so the output is the same size or slightly larger plus the cost of
running the compressor.

The denylist `ALREADY_COMPRESSED_MIME_TYPES` is a conservative set of
well-known formats. False negatives (compressing something we should
have skipped) waste CPU; false positives (skipping something we could
have compressed) waste a few percent of bytes. The denylist favours
CPU-cycle safety because the formats listed here are the ones where
gzip/zstd is reliably a net loss.

The policy applies only to `auto` mode. Explicit `'zstd'` and `'none'`
are honoured because the caller is asserting the choice. The new
`isAlreadyCompressedMimeType()` is exported for consumers that want to
make the same decision before calling `write()`.

VFS / consumer wiring (pass mimeType from VFS through to BlobStorage)
is part of the 2.5.0 #27 storage unification work — when that lands,
every media upload through VFS will engage this policy automatically.
For now consumers opt-in by passing `mimeType` in BlobWriteOptions.

Tests (1447 total, +10):
- isAlreadyCompressedMimeType helper: canonical types, case-insensitive
  + parameter stripping, false-on-missing.
- write() auto-mode: image/jpeg + video/mp4 + application/zip all skip
  compression (metadata.compression === 'none'); text/plain is allowed
  through to zstd (either 'zstd' or 'none' depending on optional dep).
- Read decompresses transparently regardless of write-side decision.
- Explicit compression options bypass the policy.
2026-05-28 11:55:10 -07:00
..
aggregation fix: deterministic code-point string collation for column store + aggregation 2026-05-26 17:35:18 -07:00
api refactor: remove src/cortex/ directory and fix README claims 2026-02-01 11:07:26 -08:00
cli feat: multi-process safety + read-only inspector mode 2026-05-15 11:25:05 -07:00
config fix: remove dead model config code for true zero-config 2025-12-18 10:31:02 -08:00
critical feat: replace transformers.js with direct ONNX WASM for Bun compatibility 2025-12-17 17:42:37 -08:00
data feat: Phase 3 - Unified Semantic Type Inference (Nouns + Verbs) 2025-10-16 10:59:26 -07:00
distributed feat(v4.0.0): Complete metadata/vector separation architecture with Azure support 2025-10-17 12:29:27 -07:00
embeddings fix: suppress misleading 'Using Q8 WASM' log when Cortex native is active 2026-03-22 14:53:48 -07:00
errors fix: find()/stats() correctness + Cortex compat (BR-FIND-WHERE-ZERO, BR-DEFENSIVE-INTERFACE) 2026-05-15 12:31:28 -07:00
graph feat: export provider contracts for the plugin surface brainy consumes 2026-05-27 14:45:40 -07:00
hnsw feat: graph link compression — delta-varint connections (2.4.0 #3) 2026-05-28 10:37:47 -07:00
import fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
importers refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
indexes/columnStore feat: column-store JS↔native interchange — raw-blob unify (2.4.0 #4) 2026-05-28 10:37:47 -07:00
integrations refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
interfaces fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
mcp refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
migration feat: add migration system with error handling, validation, and enterprise hardening 2026-02-09 16:13:14 -08:00
neural feat: array-unnest groupBy for aggregates + batch-embed entity extraction 2026-05-26 14:20:40 -07:00
patterns 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00
scripts feat: add node: protocol to all Node.js built-in imports for bundler compatibility 2025-09-17 14:20:21 -07:00
storage feat: content-type-aware compression policy in COW BlobStorage (2.5.0 #32) 2026-05-28 11:55:10 -07:00
streaming feat: Brainy 3.0 - Production-ready Triple Intelligence database 2025-09-11 16:23:32 -07:00
transaction fix: flush all native providers on shutdown to prevent data loss 2026-02-01 16:23:49 -08:00
triple fix: flush all native providers on shutdown to prevent data loss 2026-02-01 16:23:49 -08:00
types feat: exact percentile and distinctCount aggregation ops 2026-05-26 16:18:47 -07:00
universal fix: prevent fs adapter from throwing on import in browser 2025-09-17 16:22:02 -07:00
utils feat: stable EntityIdMapper — rebuild() no longer renumbers UUID→int 2026-05-28 10:37:47 -07:00
versioning fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
vfs fix: set verb.source/target to entity UUID instead of NounType 2026-02-02 09:20:38 -08:00
brainy.ts feat: graph link compression — delta-varint connections (2.4.0 #3) 2026-05-28 10:37:47 -07:00
coreTypes.ts feat(storage): add raw binary-blob primitive to every storage adapter 2026-05-27 11:50:04 -07:00
index.ts feat: add stddev/variance aggregation ops with Welford's online algorithm 2026-02-17 17:04:11 -08:00
internals.ts feat: export resolveEntityField + STANDARD_ENTITY_FIELDS from internals 2026-04-09 16:28:54 -07:00
pipeline.ts refactor: remove augmentation system and semantic type matching 2026-02-01 10:48:56 -08:00
plugin.ts feat: graph link compression — delta-varint connections (2.4.0 #3) 2026-05-28 10:37:47 -07:00
setup.ts fix: exclude __words__ keyword index from corruption detection and getStats() 2026-01-27 15:38:21 -08:00
unified.ts chore: enforce consistent coding style and semicolon removal 2025-09-29 09:50:59 -07:00
worker.ts 🧠 Brainy 2.0.0 - Zero-Configuration AI Database with Triple Intelligence™ 2025-08-26 12:32:21 -07:00