Repository navigation
fix(service-datasource): in development, a SQLite file that cannot be opened or created fails its connect instead of stepping down to an in-memory engine - #24955
Conversation
…ed is not stepped down in development resolveSqliteDriver stepped down native -> wasm -> in-memory on ANY probe failure in development, so a declared datasource on a path under a regular file (or a parent it cannot make, or a permission refusal) read connected on an in-memory engine. The step-down now applies only to a native addon that cannot load; a file-open failure (an operating-system code or SQLITE_CANTOPEN on the thrown value) is re-thrown at the native and wasm rungs, as in production. Claude-Session: https://claude.ai/code/session_011TdZ8GjixNZFLz1xJD9zzK Co-authored-by: Claude <noreply@anthropic.com>
… reaches connect() makes the file's directory before the first statement loads the addon, so the native rung already refuses every path the wasm rung would; the composite pin now says where it is refused. Claude-Session: https://claude.ai/code/session_011TdZ8GjixNZFLz1xJD9zzK Co-authored-by: Claude <noreply@anthropic.com>
…tep-down Claude-Session: https://claude.ai/code/session_011TdZ8GjixNZFLz1xJD9zzK Co-authored-by: Claude <noreply@anthropic.com>
…lite-open-failure-no-stepdown Claude-Session: https://claude.ai/code/session_011TdZ8GjixNZFLz1xJD9zzK Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check4 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 2 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin fbe205f391e285447c2a7ea3619ad1dfed615157 && git checkout fbe205f391e285447c2a7ea3619ad1dfed615157
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 865d2c8cf4accaaa5d1885d512f694e4abdce440 d353b3979bfaab76b4c0bf84af06294cddecde80 && git checkout -B drift-repro 865d2c8cf4accaaa5d1885d512f694e4abdce440 && git merge --no-ff d353b3979bfaab76b4c0bf84af06294cddecde80
node scripts/docs-audit/affected-docs.mjs --json 865d2c8cf4accaaa5d1885d512f694e4abdce440 |
席位核对 · ACCEPT ·
|
Fixes #24950
Clause-②: no
What was wrong
resolveSqliteDriver(packages/services/service-datasource/src/sqlite-driver-fallback.ts) probes a nativebetter-sqlite3connection in development and steps down to wasm SQLite, then to an in-memory engine. It stepped down on ANY probe failure. Its own module note says the step-down exists for one cause: the native addon loads lazily, so an ABI or load failure stays invisible until the first query.So under
bin/run-dev.js(NODE_ENV=development), a declared SQLite datasource whose FILE could not be opened was handed to the in-memory engine and readconnected. Measured at the public door before this change (below):os migrate apply --yeswith a boundarchivedatasource on a path under a regular file printed "neither native nor wasm SQLite available — dev falling back to InMemoryDriver", thendatasource 'archive': connected,Applied 0 change(s), and exit 0.What changed
listConnectionStates()reports itfailed. This applies to every SQLite datasource, the primary included.codeonly, never its message (isSqliteFileOpenFailure, module-private). A file failure is an operating-system code (Eplus capitals and digits, no underscore:EEXIST,ENOTDIR,ENOENT,EACCES, …) orSQLITE_CANTOPEN.SqlDriver.connect()raises these as itsSqliteDatabaseFileError.ERR_DLOPEN_FAILED,MODULE_NOT_FOUND, or no code at all, and at the probe'sSELECT 1they arrive wrapped asDATABASE_ERROR. None of these matches. Production is unchanged.@objectstack/service-datasourcepatch.Not touched:
packages/cli/**(bin/run-dev.jskeepsNODE_ENV=development),connect-failure-remedy.ts,datasource-connection-service.test.ts,packages/drivers/**. No new env var, flag or gate.Premises, measured before the change was committed
All readings were taken on this branch's base
feb09db14awith realbetter-sqlite313.0.3 on Node v22.22.0, running the base copy ofsqlite-driver-fallback.tsbyte for byte (git show BASE:path) against the real drivers.better-sqlite3: under a regular file,SqliteErrorSQLITE_CANTOPEN. A directory as the file:SQLITE_CANTOPEN. A missing parent: a code-lessTypeError("directory does not exist"). ThroughSqlDriver.connect(), which makes the directory first, the probe's thrown value isSqliteDatabaseFileErrorwith codeEEXIST(under a regular file),ENOTDIR(deeper),ENOENT(/proc/missing/),EACCES(mkdir as an unprivileged user in a 0555 directory) orSQLITE_CANTOPEN(a directory, or a new file in a read-only directory). Addon-load shapes: a realprocess.dlopenof a non-addon givesERR_DLOPEN_FAILED. A missing.nodepath givesMODULE_NOT_FOUND.bindings' "Could not locate the bindings file" has no code. A missingbetter-sqlite3package gives knex's install hint, which has no code either. No message text was needed to separate them.connect()makes the directories. Neither hits the re-throw. The standalone host already refuses a primary whose directory it cannot make (createStandaloneStack, runtime/cli: os start on an OS_DATABASE_URL sqlite file whose directory cannot be made hangs with no timeout: createStandaloneStack's synchronous recursive mkdirSync loops on ENOENT under an existing parent (the #23696 class, a second producer) #23737), and no test or flow in the tree relies on the in-memory rung for an unopenable file./proc/missing/and an EACCES mkdir, wasm failed on the same path and the in-memory rung answered (engine: memory). For a directory as the file, and for a new file in a read-only directory, wasm'sconnect()RESOLVED, so the base answer wasengine: sqlite-wasmover a file it could not read. For the directory, the first query then failed withDATABASE_ERROR(causeEISDIR). The native re-throw catches both shapes before wasm.connect()makes the file's directory before the first statement loads the addon, so even with an addon that cannot load, the native rung refuses every path the wasm rung would. I removed the wasm-rung re-throw (4da3d4bf0c). The step-down for an addon that cannot load behaves exactly as before. The wasm rung's catch only moved its best-effortdisconnect()into the sharedcloseQuietly.Pins (new file
src/__tests__/sqlite-driver-fallback-file-open-failure.test.ts)These use real
better-sqlite3, real wasm SQLite and real files in a scratch directory. The one thing faked is an addon that cannot load. The fake sits wherebetter-sqlite3loads the addon, on the knex better-sqlite3 client's ownacquireRawConnection, and throws a REAL load error: Node'sERR_DLOPEN_FAILED, orbindings' own "not built" error. The existingsrc/sqlite-driver-fallback.test.tsmocks all three driver packages and is unchanged; its 11 cases pass.SqliteDatabaseFileError:EEXIST, andENOTDIRone level deeper. No warning, never wasm or memory.SQLITE_CANTOPEN.ERR_DLOPEN_FAILEDsteps down to wasm SQLite withNATIVE_SQLITE_WASM_FALLBACK_WARNING, andSELECT 1answers.connect()refuses the file (EEXIST), as before.DatasourceConnectionService+createDefaultDatasourceDriverFactory({ dev: true }): a declaredarchive(autoConnect: true) under a regular file readsavailability: 'failed', its reason names the file, and no driver is registered.Ablation, one leg for the one behaviour changed. The native re-throw was replaced through
scripts/ablation-replace.mjs: anchor 1 → 0, marker 0 → 1, blobec4203705cea→fc817e7389d4. The subject is imported relatively fromsrc, so no dist leg applies. Predicted 4 red / 5 green, observed 4 red / 5 green: regular file, directory, addon+regular file and connection-state turned red; both absent-file pins, both addon-load pins and production stayed green. The restore was proved by blobec4203705ceaequal to HEAD and an emptygit diff HEAD.Public door, before and after
Each run is
bin/run-dev.js migrate apply --yes --database-url file:main.dbin a throwaway project whosedist/objectstack.jsondeclaresarchive(driver: 'sqlite') on a path under a regular file.dist/)05e9a8e09a)archivedatasource 'archive': connected,Applied 0 change(s), exit 0datasource 'archive': connect failed — SQLite database file '…/plain/archive.db' cannot be created: its directory '…/plain' could not be made: EEXIST …(fail-fast per ADR-0062 D5), exit 1autoConnect: true, nothing boundconnectedon the in-memory engine, exit 0datasource 'archive': connect failed — SQLite database file …(degraded, as D5 says), exit 0For the before leg, the native re-throw was mutated away in
srcthroughablation-replace(WRAP). Thenservice-datasourcewas rebuilt andablation-dist-preflightconfirmed the marker in 2 built files. After the restore, the package was rebuilt again and--absentconfirmed the marker gone from all 24 built files, with a clean tree.The card's own path,
os migrate audit-mail-bodiesover an archive datasource: NOT MEASURED. The archive examination exists only on PR #24934, which is not merged at this head. The--applyboot of that command builds its datasources through this same factory, so the after row above is that boot's mechanism.Local verification, on head
d353b3979b(after mergingorigin/mainea08ce584c, which landed #24933)pnpm exec turbo run build --filter='@objectstack/service-datasource...' --concurrency=2: 19/19 successful.pnpm --filter @objectstack/service-datasource exec vitest run --maxWorkers=2: 50 files, 892 tests passed.pnpm --filter @objectstack/service-datasource typecheck(tsc --noEmit): exit 0.--listFilesincludes the new test file.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack(no paths) derived 66 commands atd353b3979b. All 66 ran one at a time, each exit captured before any pipe: 66 × exit 0.--ranprinted: 66 derived, 66 run, 0 NOT-MEASURED, 0 UNRUN. Before the merge, the same 66 at05e9a8e09agave 65 × exit 0, andcheck:dual-build-cjs-loadsexited 3 (PREREQUISITE NOT MET: unbuilt packages elsewhere in the tree). Every family that CI runs is read from this PR's check-runs, not re-run here.Acceptance notes
SQLITE_NOTADB, which is not an open failure, so it keeps today's path. Wasm SQLite then renames the file toarchive.db.corrupt-TIMESTAMP, creates an empty database in its place, and the datasource readsconnected. The warning says "native better-sqlite3 unavailable (ABI mismatch or not built)", which is false. Measured at the same public door on this head: exit 0, the operator's file moved aside. This is outside the ruling's named route; the report puts it to the seat as an open question.os migrate plan, and the dry runs that boot withreadOnlyProbe, passsqliteAbsentFile: 'empty-in-memory'. driver-sql'sresolveSqliteAbsentFileTargetseesexistsSync(path) === falsefor a path under a regular file and opens an empty in-memory SQLite in its place. Sobin/run-dev.js migrate planwith the bound fixture above printsdatasource 'archive': connectedand plans+ os24950_note [create_table, 8 column(s)]into a file that cannot exist, exit 0. This PR does not change that: no probe failure occurs, and the fix belongs inpackages/drivers/**. The report hands it to the seat.connect()resolves over a file it cannot read (a directory as the file, a new file in a read-only directory). Its first query fails instead.applyAutoVacuumIncrementalswallows the open error, and the wasm rung's probe isconnect()alone. This is the addon-cannot-load path, which this card does not change. Carrier: none.os db cleanon a target it cannot open now prints the driver'sSqliteDatabaseFileErrortext, which names the file. Before, it printed its own "native and wasm both failed to load".provisionTelemetryDatasourcealready swallows any throw, and returnsundefinedexactly as it did for the memory engine.Generated by Claude Code