ci(test): perf and scale benchmarks leave the correctness gate
Some checks failed
CI / Node 22 (push) Successful in 12m28s
CI / Node 24 (push) Successful in 12m15s
CI / Bun (latest) (push) Successful in 12m33s
CI / Integration + conformance (Node 22) (push) Failing after 17m25s
Delta Gate / Delta gate — candidate vs control (push) Has been cancelled
Some checks failed
CI / Node 22 (push) Successful in 12m28s
CI / Node 24 (push) Successful in 12m15s
CI / Bun (latest) (push) Successful in 12m33s
CI / Integration + conformance (Node 22) (push) Failing after 17m25s
Delta Gate / Delta gate — candidate vs control (push) Has been cancelled
This commit is contained in:
parent
1fb5109351
commit
dee46b35c8
4 changed files with 102 additions and 3 deletions
|
|
@ -41,6 +41,20 @@ npm test
|
|||
Tests run on [Vitest](https://vitest.dev/). `npm test` runs the unit suite;
|
||||
see `package.json` for `test:integration`, `test:coverage`, and friends.
|
||||
|
||||
## Test gate
|
||||
|
||||
The release gate is a bare `vitest run` (no `--config` flag) — the same
|
||||
command the delta gate and CI's checks invoke. It carries the full
|
||||
correctness suite and nothing else: wall-clock/scale benchmarks
|
||||
(`tests/performance/**`, `tests/critical-performance-benchmark.test.ts`,
|
||||
`tests/api/performance-benchmarks.test.ts`) and the two tests whose outcome
|
||||
depends on the host machine or network rather than the code
|
||||
(`tests/package-size-limit.test.ts` shells out to the `npm` CLI;
|
||||
`tests/model-loading.test.ts` makes a real network call to download a model)
|
||||
are excluded from it, because a timing threshold or a flaky network call has
|
||||
no business failing a correctness check. That whole family runs on demand,
|
||||
in its own exclusive slot, via `npm run test:perf`.
|
||||
|
||||
## Standards
|
||||
|
||||
- **Strict TypeScript.** No `any` escape hatches to dodge the type checker.
|
||||
|
|
|
|||
Reference in a new issue