feat: OS-limit detection for pool-scale deployments

- New src/utils/osLimits.ts: reads RLIMIT_NOFILE (soft/hard, from
  /proc/self/limits) and vm.max_map_count at open — once per process,
  Linux-only, measurement-only — and warns loudly when either sits below the
  pool-scale floors (soft NOFILE < 65536, max_map_count < 262144), with the
  exact raise commands. On stock defaults the failure otherwise arrives as
  EMFILE or a failed mmap deep inside an index open, long after the cause
  stopped being visible. An unreadable limit produces NO warning — no
  measurement, no claim — so non-Linux platforms stay silent.
- Exported for ops doors: checkOsLimits() returns the full OsLimitsReport;
  floors exported as constants. Wired fire-and-forget in performInit after
  storage init; the check can never affect open.
- Unit tests pin the parser (incl. 'unlimited'), the floor thresholds, the
  null-never-warns rule, and the off-Linux silent path.
This commit is contained in:
David Snelling 2026-07-17 17:51:54 -07:00
parent dcd5036fe9
commit 16a73b8475
5 changed files with 246 additions and 0 deletions

View file

@ -49,6 +49,7 @@ import {
import { runGraphAudit, type GraphAuditReport } from './graph/graphAudit.js'
import { createPipeline } from './streaming/pipeline.js'
import { configureLogger, LogLevel, prodLog } from './utils/logger.js'
import { warnOnLowOsLimits } from './utils/osLimits.js'
import { setGlobalCache } from './utils/unifiedCache.js'
import type { UnifiedCache } from './utils/unifiedCache.js'
import { rankIndicesByScore, reorderByIndices } from './utils/resultRanking.js'
@ -935,6 +936,13 @@ export class Brainy<T = any> implements BrainyInterface<T> {
this.storage = await this.setupStorage()
await this.storage.init()
// OS-limit detection (once per process, Linux-only, measurement-only):
// warn NOW about RLIMIT_NOFILE / vm.max_map_count values that will bite
// at pool scale, instead of letting the operator meet them as EMFILE or
// a failed mmap deep inside an index open. Fire-and-forget — the check
// never affects open.
void warnOnLowOsLimits()
// Acquire the writer lock for filesystem (and other locking-capable) backends.
// Skipped in reader mode and on backends that don't support multi-process locking.
// Throws if another live writer holds the directory (unless force: true).