test(budgets): iron-honest wall-clock budgets — 3x the worst honest-iron measurement
Seven micro-budget tests were calibrated on one fast desktop and failed on
other honest iron with zero functional failures (bisect-proven pre-existing;
David-waived for 10.1/10.2 with this recalibration filed as the cure). Every
budget is now at least 3x the worst measurement observed across three
machines, each with a comment naming its calibration basis; the find-unified
micro-comparison of two sub-millisecond timings becomes a ratio assertion
(absolute equality of microsecond pairs can never be stable). The
inference-bound trim-history correctness test gets a timeout covering its
slowest observed run (174s) — its assertions are exact and untouched.
These remain order-of-magnitude guards; real perf enforcement lives in the
dedicated perf lanes with iron-specific budgets, per the gate-speed standard.
Known non-test artifact, documented not hidden: on slow-inference machines a
minutes-long awaited-embed loop can trip vitest's worker-RPC 60s tolerance
('Timeout calling onTaskUpdate') — all tests pass, vitest exits 1 on the
unhandled orchestration error. The CI lanes on faster iron exit clean; if a
lane ever trips it, the test moves to deterministic embeddings (its
assertions are size-bookkeeping, not embedding quality).
This commit is contained in:
parent
292e7c0406
commit
314e0e6c29
7 changed files with 46 additions and 17 deletions
|
|
@ -709,8 +709,14 @@ describe('Unified Find() Integration Tests', () => {
|
|||
|
||||
expect(simpleResult.length).toBeGreaterThan(0)
|
||||
expect(complexResult.length).toBeGreaterThan(0)
|
||||
// Simple queries should be faster
|
||||
expect(simpleDuration).toBeLessThanOrEqual(complexDuration)
|
||||
// These are both sub-millisecond operations on tiny fixture data, so
|
||||
// comparing two microsecond-scale timings for absolute equality-class
|
||||
// ordering (simple <= complex) can never be stable — timer
|
||||
// resolution and scheduling noise dominate the signal. Assert only
|
||||
// the order-of-magnitude property: the simple path isn't
|
||||
// dramatically slower than the complex one. The +5ms floor absorbs
|
||||
// noise when complexDuration itself rounds to ~0.
|
||||
expect(simpleDuration).toBeLessThanOrEqual(complexDuration * 3 + 5)
|
||||
})
|
||||
|
||||
it('should use fast paths for single search types', async () => {
|
||||
|
|
|
|||
|
|
@ -367,7 +367,9 @@ Gadget,20`
|
|||
const time = Date.now() - start
|
||||
|
||||
expect(entries.length).toBe(20)
|
||||
expect(time).toBeLessThan(5000) // < 5 seconds
|
||||
// order-of-magnitude guard: worst honest-iron measurement 8.85s
|
||||
// (32-core CPU-only box), 3x headroom
|
||||
expect(time).toBeLessThan(30000)
|
||||
console.log(` ✅ Created and copied 20 files in ${time}ms`)
|
||||
})
|
||||
})
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue