perf(8.0): bound find({ where, orderBy }) sort to the page (CTX-BR-FIND-ORDERBY)

A filtered + ordered find() materialized the FULL sorted match set before slicing
the page — passing k = filteredIds.length to the column store's filteredSortTopK.
A broad filter + orderBy returning 20 rows at billion scale produced hundreds of
millions of sorted ids to discard all but the page (O(matches) heap).

Thread the page bound (offset + limit + the hidden-tier over-fetch) into
getSortedIdsForFilter → filteredSortTopK / sortTopK / the sparse-path slice, so
the sort produces only the page. Ordering and pagination are unchanged.

From cor's 2026-06-24 co-release scaling audit (item 1).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
David Snelling 2026-06-23 15:01:49 -07:00
parent 82dde92077
commit 450084b6ce
5 changed files with 93 additions and 10 deletions

View file

@ -147,7 +147,7 @@ export interface MetadataIndexProvider {
*/
getIdSetForFilter?(filter: any): Promise<OpaqueIdSet>
getIdsForTextQuery(query: string): Promise<Array<{ id: string; matchCount: number }>>
getSortedIdsForFilter(filter: any, orderBy: string, order?: 'asc' | 'desc'): Promise<string[]>
getSortedIdsForFilter(filter: any, orderBy: string, order?: 'asc' | 'desc', topK?: number): Promise<string[]>
getFilterValues(field: string): Promise<string[]>
getFilterFields(): Promise<string[]>
getFieldValueForEntity(entityId: string, field: string): Promise<any>