Skip to content

test(plugins): run every listed column type through browse, edit and export - #3710

Merged
datlechin merged 3 commits into
mainfrom
test/3667-column-type-corpus
Oct 20, 2026
Merged

datlechin merged 3 commits into
mainfrom
test/3667-column-type-corpus

Conversation

@datlechin

@datlechin datlechin commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

  • apple/Plugins/ColumnTypeCorpus/: one runner and one corpus per engine (PostgreSQL, MySQL, MariaDB, ClickHouse, DuckDB, SQL Server, Oracle, SQLite), a column per type the driver lists, with edge values and the cell each reads as.
  • Per column the runner creates a table, inserts the values, reads them through the table tab's query and the export's stream, saves one edit through the driver's row writes (or the host's UPDATE shape), reads it back, and exports CSV, JSON and SQL. The SQL dump is replayed into the emptied table and must read back the same.
  • known marks a defect left for later with the failure it matches. Any other failure at that step fails the run, and so does a known step that passes.
  • apple/scripts/probes/check-column-type-corpus.sh <engine> builds the harness from the driver's sources and runs it against a container it starts.
  • SQLite runs its corpus in SQLiteDriverTests; project.yml adds the runner and the CSV, JSON and SQL export plugins to that target.
  • apple/scripts/ci/check-column-type-corpus.py (Repo Hygiene) fails a type a driver lists with no corpus column.
  • .claude/rules/plugin-system.md: when to run the probe.
  • Fixed, found by the corpus:
    • DuckDB SQL export wrote blobs as X'..', which DuckDB stores as the text's bytes. Now from_hex('..').
    • ClickHouse unescaped Array, Map and Tuple values, which the server writes in literal syntax: ['it\'s'] read as ['it's'], and saving that row failed.
    • ClickHouse HTTP parameters went unescaped, so a saved C:\new became C: + newline + ew, and a row holding a tab or newline could not be saved.
    • SQL Server date read with a midnight time and time with 1900-01-01.
    • SQL Server SQL export wrote '...', so text outside the database's code page restored as ??. Now N'...'.
  • Test changed on purpose: SQLExportBinaryLiteralTests.hexLiteralEnginesAreUnchanged no longer lists DuckDB.

Not in this PR

Each is a known entry in its corpus.

  • MySQL, MariaDB BIT: a typed number saves as its text's bytes (5 saves as 53); the dump writes BIT values as quoted text.
  • MySQL, MariaDB spatial: a WKT edit is bound as text and refused (1416); the dump's WKT strings are refused on restore.
  • ClickHouse: a save is a mutation the server applies after answering, so the reload can still show the old row (the probe waits for it). SETTINGS mutations_sync needs the server profile's limits and replica handling.
  • ClickHouse: a Map edit binds a String the server cannot cast (53); a JSON value never equals its read text, so a save changes nothing.
  • Oracle: dates and timestamps in a dump are text a default-NLS session refuses; an empty BLOB reads as NULL; a BFILE dumps as text; VECTOR, XMLTYPE and SDO_GEOMETRY read as placeholders.
  • SQL Server: geometry, geography and hierarchyid read as CLR hex; sql_variant reads as NULL and refuses a save; float reads with 17 digits; a column created without NULL or NOT NULL is NOT NULL on the driver's session.

Test plan

  • Probe, every engine all checks hold: PostgreSQL 17.11, MySQL 8.4.11, MariaDB 11.8.9, ClickHouse 25.8, DuckDB 1.5.2, SQL Server 2022 CU27, Oracle 23ai Free, SQLite 3.53.4.
  • Before the fixes, the same runs failed on each fixed defect above.
  • swift test: ClickHouse decoder and classifier, SQL Server date formatter.
  • Unit plan: ColumnTypeCorpusTests, JSONExportBooleanTests, SQLExportNationalLiteralTests, SQLExportBinaryLiteralTests, ClickHouseParameterBindingTests and the SQL export size and batch suites.
  • test_check_column_type_corpus.py.

Fixes #3667

…export

Fixes the export and decode defects the corpus found in JSON export, DuckDB and SQL Server dumps, ClickHouse values and parameters, and SQL Server dates.
@datlechin
datlechin enabled auto-merge (squash) October 20, 2026 17:39
…-corpus

# Conflicts:
#	apple/Plugins/JSONExportPlugin/JSONExportPlugin.swift
@datlechin
datlechin merged commit 8455484 into main Oct 20, 2026
17 of 19 checks passed
@datlechin
datlechin deleted the test/3667-column-type-corpus branch October 21, 2026 03:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test every column type per engine through browse, edit and export

1 participant