open-brainy/scripts/gate
David Snelling 1e046aa115
Some checks failed
CI / Node 22 (push) Successful in 12m14s
CI / Node 24 (push) Successful in 12m13s
CI / Integration + conformance (Node 22) (push) Failing after 13m28s
CI / Bun (latest) (push) Successful in 12m19s
ci(gate): the machine-health preflight and the truncation verdict guard
Two guards for every gate lane, born from the 2026-08-13 lost-day ledger.
gate-preflight.sh refuses a lane on a machine that cannot be trusted to
produce honest numbers — co-tenant processes named by pid and command, load
average, CPU governor, disk floors — one FATAL line per violation so the
operator can act from the message alone. vitest-verdict-check.sh refuses a
suite log that cannot be trusted as a verdict — missing or mismatched
summary counts, files that never executed (a truncated run once read as
green from three files of ninety-nine), and worker-pool death signatures.

Both verified live: the preflight correctly refuses this workstation naming
its actual offenders; the verdict guard passes/fails five fixture shapes
(clean, wrong-count, truncated, worker-death, no-summary) and both CLI
modes. Wire-up into the CI lanes rides the runner program.
2026-08-20 08:22:48 -07:00
..
gate-preflight.sh ci(gate): the machine-health preflight and the truncation verdict guard 2026-08-20 08:22:48 -07:00
README.md ci(gate): the machine-health preflight and the truncation verdict guard 2026-08-20 08:22:48 -07:00
vitest-verdict-check.sh ci(gate): the machine-health preflight and the truncation verdict guard 2026-08-20 08:22:48 -07:00

Gate Guards

Two standalone scripts that stand between a test/build gate and a false verdict: one refuses to let the gate start on a noisy machine, the other refuses to let a truncated or crashed vitest run be read as green.

Why these exist

Both guards exist because of the 2026-08-13 lost-day ledger: a gate ran on a machine under load, and separately a vitest worker pool died mid-suite while still printing a plausible-looking summary line, and in both cases the bad result was trusted and acted on for the better part of a day before anyone noticed. Neither failure mode announces itself — a loaded machine still finishes and reports numbers, and a truncated test run still prints a Test Files / Tests line — so both guards check the evidence explicitly rather than trusting that a gate finishing means the gate was valid.

gate-preflight.sh

Run before any gate lane starts. Exits 1 the moment the machine isn't gate-clean, with one FATAL: line per violation naming the exact offender (the pid and command, the path, the measured value). Prints one OK: line per check that passes. WARNING: lines mark checks that were skipped, not failures.

Checks:

# Check Default threshold Override
a 1-minute load average nproc / 2 GATE_MAX_LOAD
b any non-allowlisted process over 50% of one core 50% GATE_ALLOW_REGEX (extra pattern matched against the process's args)
c cpu0 scaling governor must be performance none (warns and skips if the sysfs path is absent)
d free space on / and /tmp 10G each GATE_SKIP_DISK_CHECK=1 to skip entirely

The allowlist for check (b) is always: this script's own process tree (its ancestors and its direct child processes), sshd, systemd, and kernel threads (recognizable by args wrapped in brackets, e.g. [kworker/0:1]). GATE_ALLOW_REGEX extends it — it does not replace it.

vitest-verdict-check.sh

Run after every vitest lane, against that lane's captured log. Fails loudly, quoting the exact line or string that tripped it, when the log's own summary can't be trusted:

  • no Test Files (or, in --count-tests mode, Tests) summary line is present at all
  • the parenthesized total in that line doesn't match what was expected
  • fewer files/tests are accounted for (passed + failed + skipped) than the total claims — a truncated run
  • the log contains Unhandled Error or Timeout calling anywhere — a dead worker pool, regardless of what the summary line claims
vitest-verdict-check.sh <log-file> <expected-file-count>
vitest-verdict-check.sh --count-tests <log-file> <minimum-test-count>

The first form checks Test Files for an exact match. The second checks Tests for a minimum (a floor, not an exact count, since the total number of individual tests moves more often than the number of test files).

Wiring into a CI lane

# Before any lane that will report a verdict:
scripts/gate/gate-preflight.sh || exit 1

# Run the suite, capturing its output:
npx vitest run tests/unit 2>&1 | tee /tmp/unit.log

# After every vitest lane, check the log against the actual file count:
EXPECTED_FILES=$(ls tests/unit/**/*.test.ts | wc -l)
scripts/gate/vitest-verdict-check.sh /tmp/unit.log "$EXPECTED_FILES" || exit 1

Exit-code contract

Script Exit 0 Exit 1
gate-preflight.sh machine is gate-clean one or more FATAL: violations printed
vitest-verdict-check.sh log's summary is trustworthy and matches usage error, missing/unreadable log, or one or more FATAL: violations printed

Non-zero from either script means: do not trust the gate that was about to run, or the result of the one that just ran.