Repository navigation
fix(driver-sql, driver-turso): SQLite $contains / $notContains / $icontains / $endsWith read the whole stored value, not only up to its first U+0000 - #20038
Conversation
…eads the whole stored value Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…ved contains/ends plans break Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…every SQLite face; respell the remote GLOB pins Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…for the SQLite stored-value cut Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check4 anchor(s) derived from 2 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 13 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e174d034001090f261b4d7e2759013cfcee98abf && git checkout e174d034001090f261b4d7e2759013cfcee98abf
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7766b62282f456ef04fd37304658b7ecd5a0fff9 667eddc476477d9b15bf05f4b18f02b684315410 && git checkout -B drift-repro 7766b62282f456ef04fd37304658b7ecd5a0fff9 && git merge --no-ff 667eddc476477d9b15bf05f4b18f02b684315410
node scripts/docs-audit/affected-docs.mjs --json 7766b62282f456ef04fd37304658b7ecd5a0fff9 |
Contract reviewServed-tier: Scope read: card #20024 (body and all comments), the PR body, and the diff Measured in the reviewer's own detached worktree at this head, which was removed afterwards. The engines are the lockfile's, read with
① Derived judgments
② Semver level
③ Boundary flagsChangeset: every sentence at head is TRUE, including:
In "What does not change", these hold:
Docblocks in PR body: TRUE where measured. The author's exact figures (292 cases, "38 of 70", "15 of 23", 2944 cells) are UNMEASURED as figures, but every direction is confirmed and the ablation counts match. Faces 3 and 4, the PG/MySQL arms, a real Turso server and mongod are correctly declared NOT MEASURED. Out-of-scope observation (not a FAIL item): the plugin-auth Open question (item 2, Implemented-by: VERDICT: PASS Isolated reviewer: a contract-review-tier subagent, fed only the card, the PR and AGENTS.md; adopted by the seat. |
…le stored value past a U+0000 (objectstack-ai#20024) (objectstack-ai#20140) Fixes objectstack-ai#20024 Clause-②: no Item 2 (ii) of objectstack-ai#20024, direction C (landing record 5823785768). On the SQLite faces, a `$like` / `$ilike` pattern without U+0000 over a stored value holding U+0000 now gets the answer `likePatternToRegexSource` gives, which is also what `@objectstack/formula` and `driver-memory` return. Item 1 landed as PR objectstack-ai#20038 and item 2 (i) as PR objectstack-ai#20124 (objectstack-ai#20041). This is the card's last item, so the PR closes it. `patch` for `@objectstack/driver-sql`, `@objectstack/driver-sqlite-wasm` (it inherits the fix) and `@objectstack/driver-turso`. Nothing that was accepted is refused now: the refusal doors, `hasNulInLikePattern` and `hasDanglingLikeEscape` are untouched. The grid, plan and cost numbers below were taken at `b424b90f7c` or earlier. Head `89e02a2521` differs from `b424b90f7c` by one changeset line only, and its emitter code is byte-identical to `7434e6646e`; every commit since then changed changeset prose only. ## The defect, measured at base `fe677aeeed` `glob()` reads the stored value as a C string, only up to its first U+0000. The grid has 59 stored values (27 holding U+0000, leading, interior, trailing, doubled, next to multi-byte and U+0001 characters, plus NULL). It runs 130 queries against them: 81 `$like` and 27 `$ilike` bare patterns, and 22 `$not` / `$or` / `$and` compositions. That is 7670 cells per face, each compared with `formula`. The four faces answered identically: | stored value | cells per face | base: differ from `formula` | head: differ from `formula` | |---|---|---|---| | holds U+0000 (26 values in the Basic Multilingual Plane) | 3380 | 359 | **0** | | no U+0000, in the Basic Multilingual Plane (28 values) | 3640 | 0 | 0 | | NULL | 130 | 0 | 0 | | outside the Basic Multilingual Plane, no U+0000 (3 values) | 390 | 20 | 20 (older than this PR, see H3) | | outside the Basic Multilingual Plane, with U+0000 (1 value) | 130 | 6 (6 against a code-point oracle) | 2 (0 against a code-point oracle) | The four faces are `SqlDriver` on better-sqlite3 (SQLite 3.53.4), `SqliteWasmDriver` on sql.js (3.49.1), `TursoDriver` local, and `TursoDriver` remote over a real `@libsql/client` `file::memory:` engine (3.45.1). At base, 74 of the 108 bare patterns got at least one row holding U+0000 wrong. Some examples: - `$like: 'a'` returned `'a'` + U+0000 + `'b'`; - `$like: ''` returned U+0000 + `'z'`; - `$like: '_'` missed a lone U+0000; - `$ilike: 'A_B'` missed `'a'` + U+0000 + `'b'`. **Base vs head, every stored value without U+0000 and NULL: 0 of 16640 cells moved, on any face.** ## What changes `likePatternPredicate`'s SQLite arm in `sql-driver.ts` now calls `sqliteLikePatternMatch`. `RemoteTransport.pushLikePattern` in `remote-transport.ts` restates the same rule for the remote transport. The PR's own parity test holds the two to the same predicate text and bindings, the precedent `pushLike` already follows. Sharing one compiler would need a new export from `driver-sql`'s `index.ts`, which is outside this claim's file surface. How a pattern compiles: - **A literal prefix followed only by `%`** (`'ab%'`, `'%'`, `'%%'`) keeps the bare `col GLOB ?` with the same binding as before. A prefix free of U+0000 cannot be changed by the cut, the same argument `$startsWith` stands on. - **Every other pattern** compiles to `CASE WHEN instr(CAST(col AS BLOB), X'00') > 0 THEN (a recursive CTE that replaces each U+0000 with one stand-in character, then GLOB) ELSE col GLOB ? END`. - A value without U+0000 takes the `ELSE` arm: the same `GLOB` as before. - NULL also takes the `ELSE` arm and stays NULL, so the NULL-safe `$not` wrapper composes unchanged. - **A case-exact pattern with a literal prefix** leads with `col GLOB 'PREFIX*'`, where PREFIX is the text before the first unescaped wildcard. Every value the pattern matches satisfies that conjunct, cut or not, and it keeps the index range. `$ilike` has no index range to keep, since `lower(col)` is not an indexed column, so it gets no conjunct. **Why the rewrite is exact.** The stand-in is the smallest code point from U+0001 up that is not a literal character of the pattern, not an ASCII letter and not a surrogate. The pattern holds no U+0000, because `hasNulInLikePattern` refuses it at every door. So a U+0000 in the value can only be matched by `%` or `_`. The stand-in is one character, as U+0000 is, and no literal position of the pattern matches it, just as none matched U+0000. Because it is not a letter, the `$ilike` fold cannot turn it into one. So the rewritten value matches exactly when the original does. **H2 is measured, not assumed.** On all three engines, `replace(v, char(0), char(1))` returns its input unchanged, `GLOB` and `LIKE` cut at U+0000, `lower()` keeps the bytes past U+0000, and `instr()` over BLOB finds U+0000. A correlated recursive CTE inside a scalar subquery works on all three and splices every U+0000, with NULL staying NULL. The earlier sentinel measurement (223 of 2392 cells wrong) is consistent with this: `replace()` never reached a U+0000 there. ## H3: characters outside the Basic Multilingual Plane under `_` (older than this PR, not widened) The spec's regex has no `u` flag, so `_` in `formula` and `driver-memory` matches one UTF-16 unit, while `GLOB`'s `?` matches one code point. At base and at head alike, 20 of 390 cells over such values without U+0000 differ from `formula` on every SQLite face (17 queries). For example, `$like: '_'` returns a stored '😀' on SQLite and not in `formula`. This PR's U+0000 arm uses `GLOB` too, so it follows the code-point reading, consistent with the arm for values without U+0000. On the astral value holding U+0000, 0 of 130 head cells differ from a code-point oracle (the spec's regex compiled with the `u` flag). The divergence is reported to the seat as an out-of-scope finding and not fixed here. ## H4: plans and cost `EXPLAIN QUERY PLAN` of the statements the real compilers emit (the `SqlDriver` compile, and the remote statement captured from a recording libSQL client) over `plan_probe(id integer primary key, v text)` with an index on `v`. Identical on better-sqlite3, sql.js and libSQL: | filter | before | after | |---|---|---| | `$like 'ab%'` | SEARCH USING COVERING INDEX (v greater-than ? AND v less-than ?) | same, same statement | | `$like 'ab_'`, `'ab%cd'`, `'abc'`, `'a%b%'` | SEARCH USING COVERING INDEX (range) | the same SEARCH, plus a correlated scalar subquery | | `$like '%ab'`, `'_b'`, `''` | SCAN | SCAN, plus a correlated scalar subquery | | `$like '%'`, `$ilike 'ab%'` | SCAN | SCAN, same statement | | `$ilike 'ab_'` | SCAN | SCAN, plus a correlated scalar subquery | | `$eq`, `$startsWith` (controls) | SEARCH | unchanged | Cost is the median of 7 runs of `count(*)` over 10,000 rows, base `GLOB` against head. This is a shared box, so read the ratios: | rows holding U+0000 | `$like '%ab%c'` | `$like '_b%'` | `$like 'ab_%'` | `$like 'ab%'` | `$ilike '%AB%C'` | |---|---|---|---|---|---| | 0% | 1.4–1.8x | 1.7–2.4x | 1.1–2.2x | 1.0x | 1.3–1.4x | | 10% | 4.2–4.7x | 5.4–7.8x | 3.3–3.7x | 0.9–1.0x | 2.4–2.8x | | 100% (1–3 each) | 25–37x (23–34 ms) | 26–44x (25–31 ms) | 23–33x (5–7 ms) | 0.7–1.1x | 12–14x (25–35 ms) | The cost for values without U+0000 is the per-row `instr()` guard. A text-`instr` spelling of it is also correct on all three engines, but it measured slower than the BLOB one. Every head count equals `formula`'s count, and base counts differ wherever U+0000 rows exist. ## H5: the remote face, executed `TursoDriver` remote over a real `@libsql/client` engine runs the whole grid in the probe (0 of 3380 cells differ at head) and the whole pin suite in CI. Its predicate text and bindings equal the local compiler's, modulo identifier quoting. A real Turso server, and its wire encoding of a string holding U+0000, are NOT MEASURED. ## H6: two pending changesets, two DELIBERATE CORRECTIONS (please confirm) This PR makes one sentence false in each of two pending changesets, and it corrects each sentence in place. One line changes in each file, and every other line of both files is byte-identical. - `Check Changeset` goes RED on both by design. `scripts/check-empty-changeset.mjs`'s foreign-changeset rule is content-blind: any `M` row on a changeset present at the merge base is refused, whichever card owns it. - Both are the **DELIBERATE CORRECTION** class. Its remedy is to confirm the correction here. ⛔ Restoring either file would put a false sentence back. 1. `.changeset/20024-sqlite-glob-stored-nul.md` is this card's own item 1 entry. The last bullet changed (line 29 of 29): - Before: "- `$like` and `$ilike` still compile to `GLOB` on SQLite, so they still read a stored value only up to its first U+0000. A pattern holding U+0000 is no longer cut there: every driver that answers `$like` now refuses it (`INVALID_FILTER` / 400), by its own entry in this release." - After: "- `$like` and `$ilike` are outside this entry. Two other entries in this release cover them: on SQLite they now read the whole stored value as well, and every driver that answers `$like` refuses a pattern holding U+0000 (`INVALID_FILTER` / 400)." 2. `.changeset/20041-like-nul-pattern-refused.md` is objectstack-ai#20041's entry. The first bullet under **Not changed here** changed (line 36 of 44). The seat's claim amendment 5830697185 on objectstack-ai#20024 (ruling A) added this bullet to this PR's surface. - Before: "- A pattern without U+0000 matched against a STORED value that holds U+0000 still differs from `formula` on the SQLite faces, because `GLOB` also reads the stored value only up to its first U+0000. No refusal of the pattern can reach that half." - After: "- A pattern without U+0000 matched against a STORED value that holds U+0000 is not refused: it is well formed, and on the SQLite faces it reads the whole stored value, by its own entry in this release." - The new sentence is true at head `89e02a2521`: - The refusal doors read only the pattern (a non-string pattern, a dangling escape, or a pattern holding U+0000). - The three pin suites pass there: 65 of 65, 22 of 22 and 23 of 23. They include patterns without U+0000 over stored values holding U+0000 on every SQLite face, and every such case returns `formula`'s rows and none is refused. - "Its own entry" is `.changeset/20024-like-stored-nul.md`. The new entry is `.changeset/20024-like-stored-nul.md`. ## Tests (at `7434e6646e`; the code is identical at head) - New suites: - `sql-driver-20024-like-stored-nul.test.ts`: 65 of 65 pass. It has 29 literal cases, each pinned and checked against `formula`, a 67-pattern grid held to `formula` cell by cell, the exact compiled statements, and an `EXPLAIN QUERY PLAN` pin. - `sqlite-wasm-20024-like-stored-nul.test.ts`: 22 of 22 pass (19 literal cases, the grid, a plan pin, and the stored-bytes premise). - `turso-20024-like-stored-nul.test.ts`: 23 of 23 pass. It runs local, remote over the stub, and remote over a real libSQL engine, plus the grid and the emitter-parity case. `driver-turso` does not depend on `formula`, so its grid is held to the spec's `matchesLikePattern`. - Full suites: - `pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2`: 187 files passed and 11 skipped; 3089 tests passed and 170 skipped. - `@objectstack/driver-sqlite-wasm`: 34 of 34 files, 644 of 644 tests. - `@objectstack/driver-turso`: 70 of 70 files, 1686 of 1686 tests. - No existing pin moved: before the new suites were added, all three full suites were green against the new emitters. - `pnpm --filter … run typecheck` exits 0 for all three packages, and `tsc --listFiles` counts each new suite once. - Ablations, each with a prediction made from the base statement or a simulation, a mutation proved on disk through `scripts/ablation-replace.mjs`, a `driver-sql` rebuild, and `ablation-dist-preflight` (marker present, then `--absent` after restore). Each restore was proved as the blob hash equal to HEAD, with `git status --porcelain` empty. - (1) Dispatch reverted in both emitters, which gives the base statement. Predicted and observed: driver-sql 25 failed / 40 passed, sqlite-wasm 15 / 7, turso 16 / 7. - (2) Stand-in fixed at U+0001 in both emitters. Predicted and observed: 4 / 61, 2 / 20, 2 / 21. The failures are the U+0001 cases, the grid, and the stand-in statement pin. - (3) Remote emitter only. Predicted and observed: turso 17 failed / 6 passed, including the parity test. ## Gates (the full union at `b424b90f7c`; the changeset gates re-run at `89e02a2521`) At `89e02a2521`, the head that adds the second correction: - `node scripts/check-empty-changeset.mjs --base origin/main` exits 1. This is expected and names both files: "This PR changes a changeset it did not add: .changeset/20024-sqlite-glob-stored-nul.md present on the merge base and CHANGED by this PR ... .changeset/20041-like-nul-pattern-refused.md present on the merge base and CHANGED by this PR". Both are the H6 DELIBERATE CORRECTIONS. - `node scripts/check-changeset-no-major.mjs --base origin/main --event`, over this body, exits 0: "This diff introduces no `major` bump.", and "LEVEL AXIS: this PR declares clause-② `no`, so no package here is declared to have grown a published surface" (declaration line `Clause-②: no`, no direction arm). - `node scripts/check-adr-0087-registration.mjs --base origin/main` exits 0: "check-adr-0087-registration: this PR adds no declared-breaking changeset (3 non-breaking changeset(s) seen)." - `node scripts/check-issue-citations.mjs --base fe677ae` exits 0: 7 citations across 2 files resolve. At `b424b90f7c`: - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derives 62 commands. All 62 were run and reconciled with `--ran`: 62 derived, 62 run, 0 NOT MEASURED, 0 UNRUN. - Every command exits 0 except `node scripts/check-empty-changeset.mjs --base origin/main`, which exits 1 for the H6 correction above. - `check:dual-build-cjs-loads`, `check:lean-entry-closure` and `check:type-check-debt` first exited 3 (PREREQUISITE NOT MET). They exit 0 after `turbo run build` over `./packages/*` and `./packages/*/*` (72 tasks). The dist-reading gates were re-run over that build and are green too. - `check:driver-conformance`: 50 covered cells, 0 DEBT, 0 exempt. - `node scripts/check-issue-citations.mjs --base fe677ae`: 7 citations resolve. - The rosters beside these paths are green: `check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`. - ESLint, narrowed: `eslint --no-inline-config --format json` over the 5 changed `.ts` files gives 5 files, 0 errors, 0 warnings, 0 ignored. - The population is every changed `.ts` file, each resolving a config under `--print-config`. - `eslint.config.mjs` never enables type-aware linting, so no untouched file's verdict can move. The full `pnpm lint` run is CI's. ## Compile surfaces 1. `driver-sql` `likePatternPredicate`, SQLite arm: TOUCHED. Measured on better-sqlite3 3.53.4. `SqliteWasmDriver` (sql.js 3.49.1) and `TursoDriver` local inherit it, and each is measured by the grid and by its own suite. The Postgres and MySQL arms are not touched and NOT MEASURED (no live server here). 2. turso `RemoteTransport.pushLikePattern`: TOUCHED. Measured on `makeLibsqlSqliteStub` and on a real `@libsql/client` engine (3.45.1). A real Turso server is NOT MEASURED. 3. and 4. service-analytics `compileScopedFilterToSql` / `lowerAnalyticsWhere`: not touched. `$like` / `$ilike` are in their unevaluated set (`preview-unevaluable-operator.test.ts`), so these faces do not compile them. Rows are NOT MEASURED. 5. `formula` `matchesFilterCondition`: not touched; it is the oracle. - Half face: objectql `having-filter`: not touched, and `$like` does not occur in `having-filter.ts`. - Thawed faces: - `driver-memory` `checkCondition`: not touched. Measured on the same grid: 0 of 7670 cells differ from `formula`. - `driver-mongodb`: not touched, NOT MEASURED (no mongod). ## Acceptance notes - H3's divergence outside the Basic Multilingual Plane is reported to the seat with its measured reach, and is not fixed here. - The objectstack-ai#20041 changeset: its line 32 ("every `$like` / `$ilike` pattern without U+0000 answers exactly as before") is scoped to objectstack-ai#20041's own change and is left as it is. The corrected line 36, in the same section, states what this PR changes for stored values. - Draft PR objectstack-ai#20104 edits `RemoteTransport` elsewhere in `remote-transport.ts`. This PR's hunk is inside `pushLikePattern` only. Written by `session_01Bvd69VPa6puiNzzPUroDBx` (os-dev, `mode:subagent`) for the `domain:engine` seat. --------- Co-authored-by: Claude <noreply@anthropic.com>
Part of #20024
Clause-②: no
What this changes
This takes item 1 of #20024, the stored value's cut. Item 2 (
$like/$ilike) stays open on #20024 for a decision; see "Item 2" below.On the SQLite faces,
$contains,$notContains,$icontainsand$endsWithcompiled toGLOBfor a comparand without U+0000. SQLite'sglob()reads the stored value as a C string, only up to its first U+0000. So$contains: 'b'missed a value stored as'a'+ U+0000 +'b',$endsWith: 'a'returned it, and$icontains: 'B'missed'A'+ U+0000 +'B'. Nothing raised. #19999 (PR #20019,9d81af714f) had already moved a comparand that HOLDS U+0000 onto length-aware constructs; this sends everycontains/endscomparand there, which is option A of contract review 5821310673 (reconfirmed in 5822054158).contains($contains,$notContains,$icontains)instr(col, ?) > 0ends($endsWith), non-empty comparandcoalesce(substr(CAST(col AS BLOB), -length(CAST(? AS BLOB))), CAST(col AS BLOB)) = CAST(? AS BLOB)ends, empty comparandinstr(col, ?) > 0starts($startsWith), comparand without U+0000col GLOB ?, unchanged byte for bytestarts, comparand holding U+0000instr(col, ?) = 1, unchanged from #19999$endsWithcomparand needed its own route. In the suffix construct-length('')is-0, whichsubstrreads as "from the start", so the construct would answer false for'abc'. Measured on all three engines: the suffix construct with''answers 0 on'abc', whileinstr('abc', '')andinstr('', '')are both 1 andinstr(NULL, '')is NULL, which is whatGLOB '*'answered. An empty string is a suffix of every value, so it takesinstr().$startsWithkeepsGLOB. A prefix free of U+0000 lies wholly before the value's first U+0000, or it is not a prefix at all, so the value's cut cannot change that answer (0 of 14$startsWithprobe cases differed at base, including values that start with U+0000). It is also the one text construct an index serves; see H3.lower()on both sides, ASCII-only. The negation is stillNOT (...)inside the unchanged NULL-safe$notContainswrapper. Nothing is escaped, becauseinstr()and the BLOB suffix have no pattern language, so*,?and[are bound as written.textMatchPredicate/sqliteLengthAwareTextMatchinpackages/drivers/driver-sql/src/sql-driver.ts, andRemoteTransport.pushLikeinpackages/drivers/driver-turso/src/remote-transport.ts.SqliteWasmDriverandTursoDriver's local mode inherit the first. No refusal door, nopackages/specfile and no shared conformance table is touched.Measured at base
9d81af714(hypotheses H1, H2, H5)A 292-case probe drove the public
findof six faces over 17 stored values (U+0000 at start, middle and end, alone and doubled, multi-byte UTF-8 around it, GLOB and LIKE metacharacters behind it,''and NULL), on dists built at base:SqlDriveron better-sqlite3 (SQLite 3.53.4);SqliteWasmDriveron sql.js (3.49.1);TursoDriverlocal;TursoDriverremote over a real@libsql/client:memory:engine (SQLite 3.45.1);InMemoryDriver, and@objectstack/formulamatchesFilterConditionas the oracle.The cases were 5 text operators times 19 comparands (14 without U+0000, 5 with), each bare and under
$not; 36$likeand 9$ilikepatterns, each bare and under$not; and 12$not/$or/$andcompositions.formulaat base$not$not$likeleg$like/$ilike, bare and under$not$contains8 of 14,$notContains8 of 14,$icontains9 of 14,$endsWith13 of 14. The four SQLite faces gave byte-identical answer lists on all 292 cases, at base and at head.$startsWithwith a comparand without U+0000: 0 of 14 differ, bare and under$not, at base and at head.$icontains: 'É'returns'CAFÉ'+ U+0000 and$icontains: 'é'does not; the fold stayslower()on both sides.InMemoryDriverequalsformulaon 292 of 292 at base and at head (the control). No face raised on any case, at base or at head.The constructs themselves were measured with SQL literals, no bind, over 16 comparands without U+0000 (including
*?[y,%,_, U+0001 and multi-byte ones), 23 stored values and both folds:instr() > 0, the BLOB suffix, andNOTover each, three-valued against JavaScriptincludes/endsWith(NULL expected on NULL). That is 2944 cells per engine and 0 disagreements on each of better-sqlite3 3.53.4, sql.js 3.49.1 and libSQL 3.45.1.H3: the plans
EXPLAIN QUERY PLANof the SQLSqlDrivercompiles, run overcreate table plan_probe (id integer primary key, v text)with an index onv, on each engine:GLOB)$contains: 'ab'SCAN plan_probeSCAN plan_probe(instr)$notContains: 'ab'SCANSCAN$icontains: 'Ab'SCANSCAN$endsWith: 'ab'SCANSCAN(BLOB suffix)$startsWith: 'ab'SEARCH plan_probe USING COVERING INDEX plan_probe_v (v>? AND v<?)$like: 'ab%'SEARCH … USING COVERING INDEX plan_probe_v (v>? AND v<?)$eq: 'ab'(control)SEARCH … (v=?)Identical on better-sqlite3, sql.js and libSQL. H3 holds: the four moved shapes scanned under
GLOBalready, so no index plan is lost. The remote transport's statements were not put throughEXPLAIN; they are the same constructs, pinned by text inturso-20024-glob-stored-nul.test.ts.Item 2 (
$like/$ilike) is not in this PRMeasured, and left for a decision on #20024:
GLOBandLIKEboth stop at U+0000. The way around it I tried first, substituting a sentinel for U+0000 in the value and the pattern, is dead: SQLite'sreplace()treats a search string whose first byte is 0 as empty and returns its input unchanged.hex(replace('a' || char(0) || 'b', char(0), char(1)))is610062on all three engines. Over a grid of 52 patterns, 23 values and both folds (2392 cells per engine),GLOBdisagrees with@objectstack/specmatchesLikePatternon 294 cells and the sentinel construct still on 223.instr()does not apply, because the pattern has wildcards, and_is one character, so a byte-level BLOB program would have to decode UTF-8 lengths.$likepatterns differ fromformulaon every SQLite face. For example,$like: 'a'returns'a'+ U+0000 +'b', and$like: ''returns U+0000 +'z'. No refusal can reach this half, because it depends on stored data.$likepattern is one spec predicate (hasDanglingLikeEscape) applied at every face's refusal door. The same shape for U+0000 needspackages/specand the refusal doors that engine + driver-sql: seven filter-refusal doors ignore the #8220 provenance mark, so a refusal inside apolicyread-scope subtree relays the policy field (and some comparands) in its 400: the #7929 redaction is not applied #20020 is working in, and what Postgres and MySQL do with such a pattern is not measured here (no live server). A SQLite-only refusal would make SQLite the one face that refuses whatdriver-memoryanswers.The options, with the measured costs, are in the
os-dev-reporton #20024.Compile surfaces (
references/compile-surfaces.md, re-verified with its grep at this head)driver-sqltextMatchPredicateSQLite arm (sql-driver.ts)sql-driver-20024-glob-stored-nul.test.ts(better-sqlite3).driver-sqlite-wasminherits it and is pinned bysqlite-wasm-20024-glob-stored-nul.test.ts;driver-tursolocal mode byturso-20024-glob-stored-nul.test.ts.driver-sqllikePatternPredicateSQLite armRemoteTransport.pushLike(remote-transport.ts)makeLibsqlSqliteStubAND on a real@libsql/clientfile::memory:engine, beside local mode, all three to the JavaScript rows. A real Turso server and its wire encoding of U+0000 are NOT MEASURED.RemoteTransport.pushLikePatterncompileScopedFilterToSqllowerAnalyticsWhereformulamatchesFilterConditionapplyHavingformula;$icontains: ''is refused, the pre-existing refusal driver-sql shares.driver-memorycheckConditionformula, at base and at head.driver-mongodbtranslateFieldOperatorsTests
sql-driver-20024-glob-stored-nul.test.ts(74 tests),sqlite-wasm-20024-glob-stored-nul.test.ts(24),turso-20024-glob-stored-nul.test.ts(26, three transports per case). The driver-sql and sqlite-wasm suites also check every case againstformula; the turso cases are a subset of the driver-sql rows. They cover the four moved operators,$startsWith(H2), the empty comparand, and$not/$or/$andover each moved shape with the''and NULL rows in every answer set. The compile block pins the new SQL and raw bindings, the empty-$endsWithroute, and$startsWith'sGLOBplus its escaped pattern byte for byte.pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2at667eddc47: 183 files passed, 11 skipped (live-dialect cells); 2811 tests passed, 170 skipped.driver-turso65 of 65 files, 1471 of 1471 tests, anddriver-sqlite-wasm32 of 32 files, 606 of 606 tests, atcf0590562, whose code is byte-identical to667eddc47(the one later commit adds only the changeset).typecheck(tsc --noEmit) exits 0 in all three packages, andtsc --listFilesshows each program includes its new suite.--no-inline-config --format jsonover the 15 changed TS files reports 15 files, 0 errors, 0 warnings, none ignored (every file resolves a config through--print-config).eslint.config.mjsnever enables type-aware linting (noparserOptions.project, as its own comment says), so this diff cannot move the verdict of a file it does not touch. The fullpnpm lintis CI's.Respelled assertions (markers the moved plans broke)
driver-sql/src/sql-driver-19999-glob-nul-comparand.test.ts$contains 'a*b'compiles to`v` GLOB ?with*a[*]b*;$endsWithbinds*ab;$icontainscompileslower(v) GLOB lower(?);$notContainsmatchesNOT GLOB$startsWithkeeps`v` GLOB ?withab*; the other four compile withoutGLOBand binda*brawdriver-sql/src/sql-driver-15683-temporal-text-operator-gate.test.tstoMatch(/LIKE|GLOB/); negatives withoutinstr(toMatch(/LIKE|GLOB|instr\(/);instr\(added to both negativesdriver-sql/src/sql-driver-17343-multi-valued-boolean-membership.test.tsREAL_PREDICATE.sqlite/LIKE|GLOB|json_each\(/; controltoMatch(/LIKE|GLOB/); negative withoutinstr(/LIKE|GLOB|instr\(|substr\(CAST\(|json_each\(/; control/LIKE|GLOB|instr\(/;instr\(added to the negativedriver-sql/src/sql-driver-17590-json-column-membership.test.tstoMatch(/LIKE|GLOB/)andtoMatch(/GLOB/); rest-of-familytoMatch(/LIKE|GLOB/); "not a pattern match"not.toMatch(/LIKE|GLOB/)/LIKE|GLOB|instr\(/,/instr\(/,/LIKE|GLOB|instr\(|substr\(CAST\(/, and the negative gainsinstr\(|substr\(CAST\(driver-sql/src/sql-driver-icontains-and-retired-operators.test.tslower(name) GLOB lower(and*acme*;name GLOBinstr(lower(name), lower('acme')) > 0;instr(name, 'acme') > 0; neither containsGLOBdriver-sql/src/sql-driver-text-case-conformance.test.ts$containstoMatch(/GLOB/),$icontainslower(…) GLOB lower(; numeric-column controltoMatch(/LIKE|GLOB/)$containsinstr(,$icontainsinstr(lower(…), lower(,$startsWithGLOB; control/LIKE|GLOB|instr\(/;instr\(added to both negativesdriver-turso/src/turso-19999-glob-nul-comparand.test.ts$contains 'a*b'sends"v" GLOB ?$startsWith 'a*b'sends"v" GLOB ?driver-turso/src/remote-transport-text-predicates.test.tslower("name") GLOB lower(?);"name" GLOB ?;$contains*/?/[/a*bbind*[*]*,*[?]*,*[[]*,*a[*]b*instr(lower("name"), lower(?)) > 0;instr("name", ?) > 0; the escape class is pinned on$startsWith([*]*,[?]*,[[]*,a[*]b*), plus a new case that$contains 'a*b'bindsa*brawdriver-turso/src/remote-transport-null-comparand-refusal.test.ts$contains: truebinds*true*truedriver-turso/src/remote-transport-undefined-comparand-refusal.test.ts$contains: 'w'binds*w*wBefore-red and ablations (one-off; no test left behind)
Both ran from committed state through
scripts/ablation-replace.mjs(anchor counts and blob hashes checked on disk, restore proven as blob == HEAD withgit diff HEADempty), withdriver-sqlrebuilt andscripts/ablation-dist-preflight.mjsconfirming the mutation indist/before the suites ran, becausedriver-sqlite-wasmand turso's local mode read that dist.if (shape !== 'starts' || text.includes(NUL_CHARACTER))back to base'sif (text.includes(NUL_CHARACTER))), which is base behaviour. Predicted by simulatingGLOB's cut over each suite's cases, then observed exactly: driver-sql 24 failed and 50 passed of 74 (22 row cases plus the moved-shapes and empty-$endsWithcompile pins; the$startsWithpin stayed green); sqlite-wasm 16 failed and 8 passed of 24; turso 18 failed and 8 passed of 26. In the 12 turso row diffs whose three columns printed in full,local,remoteandremoteLibsqlwere all wrong the same way; the base probe had already found local mode and remote-over-libSQL byte-identical to the other SQLite faces on all 292 cases.shape === 'ends' && text !== ''toshape === 'ends', both emitters). Predicted and observed: driver-sql 3 failed of 74 ($endsWith '',$not $endsWith '', the empty-$endsWithcompile pin), sqlite-wasm 2 of 24, turso 3 of 26.git status --porcelainempty,driver-sqlrebuilt, and preflight reads the guard back indist/with the tree clean.Gates (at
667eddc47)node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 62 commands from the real diff, the same 62 the PM list named. All 62 were run.check:dual-build-cjs-loads,check:lean-entry-closureandcheck:type-check-debtfirst answeredPREREQUISITE NOT MET(exit 3). They answered 0 afterturbo run build --filter='./packages/*' --filter='./packages/*/*'(72 tasks).check:dts-closure,check:published-filesandcheck:sourcemap-no-sources-contentwere re-run over that build and answered 0.--ran: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN".check-changeset-no-major: "This diff introduces nomajorbump."check-adr-0087-registration: "this PR adds no declared-breaking changeset (1 non-breaking changeset(s) seen)."node scripts/check-issue-citations.mjs --base 9d81af714: 12 citations across 2 files, all resolve.pnpm check:driver-conformancereads the same before (on agit archiveof9d81af714) and after:50 covered cell(s), 0 in the DEBT ledger, 0 exempt.check-changeset-fixed,check:authz-resolver,check:error-code-casing,check:filter-alias-parity.pnpm check:nul-bytespasses, and a control-byte scan of the changed files finds none. U+0000 is spelledString.fromCharCode(0x00)throughout.Acceptance notes (not filed; for the seat)
GLOBstill cuts a STORED value at its first U+0000, so a NUL-free$contains/$notContains/$icontains/$endsWithanswers wrongly on it; a$like/$ilikepattern holding U+0000 is cut the same way #20024 is open and needs a decision (above; options in theos-dev-report).packages/plugins/plugin-auth/src/auth-contains-filter.test.tssays in its docblock that the SQL family "emitsGLOBon the SQLite dialects now". For$containsthat isinstr()after this PR. It is prose in a test outside this claim; no assertion reads it.Generated by Claude Code