fix: code-point string collation in LSM SSTable, COW trees/refs, sorted queries
Replaces localeCompare / raw relational operators with compareCodePoints (UTF-8 byte order) in the remaining persisted or native-facing comparison sites: - SSTable sourceId sort + zone-map range check + binary search (graph LSM) - COW TreeObject + RefManager name sorts (reproducible content hashes and ref listings across environments) - getSortedIdsForFilter (numeric-aware; code-point for strings) so the JS sort fallback matches the native column store exactly Deterministic across OS/Node/ICU and byte-identical to the native engine. No migration: COW serialize/deserialize preserve stored order, so existing trees keep their hashes; only new writes adopt code-point order.
This commit is contained in:
parent
547721ae14
commit
7493d8e33f
4 changed files with 29 additions and 8 deletions
|
|
@ -20,6 +20,7 @@
|
|||
*/
|
||||
|
||||
import type { COWStorageAdapter } from './BlobStorage.js'
|
||||
import { compareCodePoints } from '../../utils/collation.js'
|
||||
|
||||
/**
|
||||
* Reference type
|
||||
|
|
@ -226,7 +227,7 @@ export class RefManager {
|
|||
// Mark cache as valid
|
||||
this.cacheValid = true
|
||||
|
||||
return refs.sort((a, b) => a.name.localeCompare(b.name))
|
||||
return refs.sort((a, b) => compareCodePoints(a.name, b.name))
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@
|
|||
*/
|
||||
|
||||
import { BlobStorage } from './BlobStorage.js'
|
||||
import { compareCodePoints } from '../../utils/collation.js'
|
||||
|
||||
/**
|
||||
* Tree entry: name → blob hash mapping
|
||||
|
|
@ -98,7 +99,12 @@ export class TreeBuilder {
|
|||
*/
|
||||
async build(): Promise<string> {
|
||||
const tree: TreeObject = {
|
||||
entries: this.entries.sort((a, b) => a.name.localeCompare(b.name)),
|
||||
// Code-point (UTF-8 byte) order so the serialized tree — and therefore its
|
||||
// content hash — is reproducible across environments (localeCompare's default
|
||||
// locale varies by OS/Node/ICU). Sorting happens only here at write time;
|
||||
// deserialize() preserves stored order, so existing trees keep their hashes
|
||||
// and stay valid — no migration needed.
|
||||
entries: this.entries.sort((a, b) => compareCodePoints(a.name, b.name)),
|
||||
createdAt: Date.now()
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue