soulcraftlabs/open-brainy
Archived
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 345 Packages Wiki Activity

ci(test): perf and scale benchmarks leave the correctness gate
Some checks failed
CI / Node 22 (push) Successful in 12m28s
CI / Node 24 (push) Successful in 12m15s
CI / Bun (latest) (push) Successful in 12m33s
CI / Integration + conformance (Node 22) (push) Failing after 17m25s
Delta Gate / Delta gate — candidate vs control (push) Has been cancelled

Browse source
This commit is contained in:
David Snelling 2026-09-02 10:17:05 -07:00
parent 1fb5109351
commit dee46b35c8
4 changed files with 102 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

33
vitest.config.ts
View file

@ -2,9 +2,16 @@ import { defineConfig } from 'vitest/config'
/**
* Vitest Configuration - Optimized for Memory-Intensive Tests
*
*
* Handles ONNX transformer model testing (4-8GB memory requirement)
* Based on 2024-2025 best practices
*
* THE CORRECTNESS GATE: this is the config a bare `vitest run` (no
* `--config` flag) picks up — the delta gate and CI both invoke it that
* way. See CONTRIBUTING.md's "Test gate" section for the full picture.
* Wall-clock/scale benchmarks and tests whose outcome depends on the host
* machine or network rather than the code are excluded below and run on
* demand instead, in their own slot: `npm run test:perf`.
*/
export default defineConfig({
test: {
@ -38,7 +45,29 @@ export default defineConfig({
'node_modules/**',
'dist/**',
'scripts/**',
'**/*.browser.test.ts'
'**/*.browser.test.ts',
// Wall-clock/scale benchmark family — timing assertions and scale
// sweeps whose pass/fail depends on the host machine's speed, not on
// the code. Whole files only (a file that mixes correctness describes
// with a perf describe stays in the gate). Run on demand via
// `npm run test:perf`, which targets exactly this list.
'tests/performance/**',
'tests/critical-performance-benchmark.test.ts',
'tests/api/performance-benchmarks.test.ts',
// Environment-dependent by construction, not timing-based:
// package-size-limit shells out to the `npm` CLI (not guaranteed
// present — the functional gate lane is Bun-only host-mode with no
// Node.js runtime) and parses npm-version-specific `npm pack` notice
// text; model-loading's "Real Model Download Integration" case makes
// a genuine, unmocked network call to HuggingFace (its own header
// says "Uses REAL transformer models - NO MOCKING"), and the whole
// file imports `../src/embeddings/model-manager.js`, which no longer
// exists anywhere under src/ — neither belongs in a gate that must be
// deterministic.
'tests/package-size-limit.test.ts',
'tests/model-loading.test.ts'
],
// REPORTERS: Dot for CI, verbose for local