fix: aggregation state adoption on reopen + single-flight backfill + query-cap ratchet removal
- AggregationIndex: defineAggregate before init no longer forces a backfill. init reconciles instead of clobbering: the app definition wins, persisted state is adopted on hash match, a write landing pre-adoption forces an exact rescan, and a successful state load clears the backfill flag. New ready() settles every adoption decision before query paths consult backfill state. - brainy: backfills are single-flight and batched. Concurrent queries share one store walk and every pending aggregate fills from that same walk; the old behavior let each concurrent query wipe the others' partial state and start its own full walk, so a store under steady aggregate traffic never converged. queryAggregate also waits for persisted definitions before deciding an aggregate does not exist. - paramValidation: recordQuery is telemetry-only. The duration-based cap ratchet (x0.8 per recorded query while lifetime-average exceeded 1s, floored at 1000 - below the documented 10000 auto floor, reported under a stale basis label) is removed; the cap comes from its construction-time basis or explicit overrides alone. - storage: __aggregation_* and singleton system keys (brainy:entityIdMapper) are recognized before the unknown-key warning fires; routing is unchanged. - docs: find-limits cap-immutability note, aggregation reopen/adoption semantics, RELEASES.md 8.5.1 entry.
This commit is contained in:
parent
593bb8b0f9
commit
da55be7520
10 changed files with 584 additions and 44 deletions
|
|
@ -270,27 +270,24 @@ describe('Zero-Config Parameter Validation', () => {
|
|||
expect(config.availableMemory).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it('should adapt limits based on query performance', () => {
|
||||
const initialConfig = getValidationConfig()
|
||||
const initialLimit = initialConfig.maxLimit
|
||||
|
||||
// Simulate fast queries with large results
|
||||
it('never mutates the cap from query timing (telemetry only)', () => {
|
||||
const initialLimit = getValidationConfig().maxLimit
|
||||
|
||||
// Fast queries with large results: no silent growth.
|
||||
for (let i = 0; i < 10; i++) {
|
||||
recordQueryPerformance(50, initialLimit * 0.9)
|
||||
}
|
||||
|
||||
const updatedConfig = getValidationConfig()
|
||||
// Limit might increase if performance is good
|
||||
expect(updatedConfig.maxLimit).toBeGreaterThanOrEqual(initialLimit)
|
||||
|
||||
// Simulate slow queries
|
||||
for (let i = 0; i < 10; i++) {
|
||||
recordQueryPerformance(2000, 100)
|
||||
expect(getValidationConfig().maxLimit).toBe(initialLimit)
|
||||
|
||||
// A burst of catastrophically slow queries must not strangle the cap.
|
||||
// The removed "learning" ratchet shrank it 20% per recorded query down
|
||||
// to a floor of 1000 — below the documented MIN_AUTO_QUERY_LIMIT — and
|
||||
// the error message blamed "available free memory" (a production
|
||||
// incident: every find({ limit: 5000 }) failed on an idle 23GB-free box).
|
||||
for (let i = 0; i < 50; i++) {
|
||||
recordQueryPerformance(90_000, 100)
|
||||
}
|
||||
|
||||
const finalConfig = getValidationConfig()
|
||||
// Limit should decrease if performance is poor
|
||||
expect(finalConfig.maxLimit).toBeLessThanOrEqual(updatedConfig.maxLimit)
|
||||
expect(getValidationConfig().maxLimit).toBe(initialLimit)
|
||||
})
|
||||
})
|
||||
})
|
||||
Loading…
Add table
Add a link
Reference in a new issue