[autoCAT6]

Which issue does this PR close?

Closes #25662.

Rationale for this change

External catalogs and table providers need to reuse credentials and allow applications to supply them outside SQL. Passing each credential through connection options exposes values to statement and plan formatting and leaves each integration to manage its own store.

What changes are included in this PR?

• Add a pluggable asynchronous SecretsStore, an atomic in-memory default, and session/builder access for providers and their factories.
• Add CREATE [OR REPLACE] SECRET [IF NOT EXISTS] and DROP SECRET [IF EXISTS], using the same store as programmatic operations. Secrets are generic string option collections; providers interpret their fields and named references.
• Redact secret statement/plan formatting, reject unsupported native secret wrappers without printing values, and refuse to serialize credential-bearing CREATE SECRET DDL.
• Add a runnable two-provider example and binary/JSON connection-plan tests. The receiver resolves a name against its own store; the producer's credential is not transported with the plan.

What is the testing strategy for this PR?

Added 27 Rust tests plus secrets.slt, covering the store, SQL parser/planner, session execution, and protobuf/JSON connection-plan boundaries. Validation on Rust 1.99.1:

• Extended workspace lib/tests/bins: 12,817 passed, 8 ignored; 537/537 SQL logic files completed. Excluded datafusion-examples, datafusion-benchmarks, and datafusion-cli, as in the contributor instructions; enabled avro,json,backtrace,extended_tests,recursive_protection,parquet_encryption.
• Focused SQL/store/serialization tests, six public-API provider tests, and the binary/JSON reference roundtrip passed. The six store unit tests were rerun against the final test source.
• cargo run --locked --profile ci -p datafusion-examples --example data_io -- secrets passed.
• cargo fmt --all -- --check and cargo clippy --locked --profile ci --all-targets --all-features --keep-going -- -D warnings passed.
• Generated the example index and ran the pinned Markdown formatter; TOML, spelling, workflow, file-size and unused-dependency checks passed.

Existing sql_planner benchmarks compared against f9bc5a3fc60cc7f4319cd2c156cb3d024a8c1b21, with independent target directories and the normal optimized release/LTO profile. Initial baseline/fixed/baseline/fixed runs used 20 samples, 1 s warm-up and 3 s measurement per case. Times below are microseconds per iteration; TPC-H is 21 schema-only logical plans.

Case	Baseline runs (µs)	New implementation runs (µs)
logical_select_one_from_700	130.053 / 126.554	121.485 / 125.088
logical_wide_filter_200_predicates	784.394 / 703.979	915.057 / 732.920
logical_plan_tpch_all	2324.694 / 2332.044	2276.311 / 2355.350

The initial wide-filter difference warranted a longer check with the same binaries, 5 s warm-up, 8 s measurement and 30 samples. Other task builds were paused for both sets of measurements:

Order	Variant	Point estimate (µs)	Criterion CI (µs)
1	New	684.074	681.104–687.737
2	Baseline	688.689	683.534–696.559
3	Baseline	678.172	675.996–680.448
4	New	685.348	682.462–688.701

The initial large gap did not reproduce; this local check establishes neither a performance improvement nor universal absence of regressions. Compiled production sources were unchanged during measurement; subsequent edits to the session unit-test iterator and a separate protobuf integration test were excluded from this benchmark target.

Tests include duplicate/replacement/removal semantics, concurrent creates/removes, session sharing/isolation, custom stores, wrong/missing credentials, DDL restrictions, canary redaction, native parser compatibility, and reference-only plan roundtrips.

Are there any user-facing changes?

Yes: new session APIs and CREATE/DROP SECRET SQL, documented with an example and migration notes. DdlStatement and SQL parser Statement gain CreateSecret and DropSecret variants; downstream exhaustive matches must handle them. The Session trait method has a default implementation so existing implementors continue to compile. The default store is session-scoped plaintext memory; persistence, authorization, external secret-manager integrations, and automatic object-store configuration are outside this initial implementation. Names/references remain visible, raw SQL can still be retained by callers, and removing a secret does not revoke existing provider connections or snapshots.