fix(storage): resolve persistence restart bug across all storage adapters

Critical bug fix that restores data persistence after application restart for all storage adapters (GCS, S3, OPFS, FileSystem).

**Root Cause:**
Storage adapters were loading entity counts from _system/counts.json but not returning totalCount in pagination methods, causing rebuildIndexesIfNeeded() to see undefined and incorrectly assume no data existed.

**Changes:**
- GCS Storage: Return totalCount in getNodesWithPagination() and getVerbsWithPagination()
- S3 Storage: Call initializeCounts() in init() and return totalCount in pagination methods
- OPFS Storage: Call initializeCounts() in init() to load pre-calculated counts
- BaseStorage: Add defensive validation to prevent future adapters from missing totalCount

**Impact:**
- Production deployments now work correctly (Cloud Run, AWS, Kubernetes)
- Data persists correctly across container restarts
- Serverless scale-to-zero deployments now functional
- All storage adapters use O(1) pre-calculated counts (scales to millions of entities)

**Testing:**
- Build passes with no TypeScript errors
- All existing tests pass
- Defensive validation ensures future storage adapters won't have this bug

Fixes critical production issue affecting all persistent storage deployments.
This commit is contained in:
David Snelling 2025-10-09 15:07:18 -07:00
parent c502b56bb4
commit 01e89fffca
4 changed files with 40 additions and 17 deletions

View file

@ -508,9 +508,21 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// This prevents pagination bugs from causing infinite loops
const safeHasMore = items.length > 0 ? result.hasMore : false
// VALIDATION: Ensure adapter returns totalCount (prevents restart bugs)
// If adapter forgets to return totalCount, log warning and use pre-calculated count
let finalTotalCount = result.totalCount || totalCount
if (result.totalCount === undefined && this.totalNounCount > 0) {
console.warn(
`⚠️ Storage adapter missing totalCount in getNounsWithPagination result! ` +
`Using pre-calculated count (${this.totalNounCount}) as fallback. ` +
`Please ensure your storage adapter returns totalCount: this.totalNounCount`
)
finalTotalCount = this.totalNounCount
}
return {
items,
totalCount: result.totalCount || totalCount,
totalCount: finalTotalCount,
hasMore: safeHasMore,
nextCursor: result.nextCursor
}
@ -705,9 +717,21 @@ export abstract class BaseStorage extends BaseStorageAdapter {
// This prevents pagination bugs from causing infinite loops
const safeHasMore = items.length > 0 ? result.hasMore : false
// VALIDATION: Ensure adapter returns totalCount (prevents restart bugs)
// If adapter forgets to return totalCount, log warning and use pre-calculated count
let finalTotalCount = result.totalCount || totalCount
if (result.totalCount === undefined && this.totalVerbCount > 0) {
console.warn(
`⚠️ Storage adapter missing totalCount in getVerbsWithPagination result! ` +
`Using pre-calculated count (${this.totalVerbCount}) as fallback. ` +
`Please ensure your storage adapter returns totalCount: this.totalVerbCount`
)
finalTotalCount = this.totalVerbCount
}
return {
items,
totalCount: result.totalCount || totalCount,
totalCount: finalTotalCount,
hasMore: safeHasMore,
nextCursor: result.nextCursor
}