[dimitri-yatsenko]

Closes #1627.

Summary

database.backend now defaults to "postgresql" on port 5432. MySQL remains a fully supported backend, selected with DJ_BACKEND=mysql or database.backend = "mysql". A connection that fails while the backend is still the default says so:

<original error>
The default backend is postgresql since DataJoint 2.4; set database.backend='mysql' (DJ_BACKEND=mysql) for a MySQL server.

Changes

• Default backend (settings.py). backend defaults to "postgresql", and the docstring examples and field descriptions match.
• The port follows the backend unless it is set.
  • Before: the port was derived only when the settings were first built. A backend set later, in datajoint.json or by assignment, kept the other backend's port. With the new default, {"database": {"backend": "mysql"}} would have connected to 5432.
  • Now: the validator recomputes a port that was not set whenever the backend changes. A port set by DJ_PORT, datajoint.json, or assignment is kept. Setting it to None returns it to the backend default.
  • The test that pinned the old behavior (test_port_auto_detection_on_initialization) is replaced.
• Hint on failure (connection.py). The hint applies when the backend is still the default: no backend argument was passed, and database.backend is not in the settings' model_fields_set. In that case, a failure to create the adapter or to connect is re-raised as a DataJointError that keeps the original message, appends the hint, and chains the original error. The adapter failure covers a missing psycopg2.
  • A backend chosen by DJ_BACKEND, datajoint.json, dj.config assignment, the backend argument, or Instance(backend=...) raises the original error unchanged.
  • Reconnects are not affected.
• Instance passes no backend argument to Connection. Passing its config's value made every backend look chosen, so Connection now reads the backend from that config.
• save_template writes "backend": "postgresql" and "port": 5432.
• PostgreSQL first:
  • The CLI help and docstring examples, plus a note to set DJ_BACKEND=mysql for MySQL.
  • The Connection and Instance docstrings.
  • docker-compose.yaml: service order, depends_on, and header comments.
  • CONTRIBUTING.md: test descriptions, external-container recipes, and environment variable tables.
• Compose app service. Its DJ_HOST/DJ_USER/DJ_PASS name the MySQL service, which the test suite reads as its MySQL server. The service now sets DJ_BACKEND=mysql to match.

Tests that relied on MySQL being the default

• tests/conftest.py:
  • A new session fixture, mysql_backend_config, selects MySQL in dj.config for every test that connects through db_creds_root/db_creds_test, as dj.conn(**db_creds_root) and dj.Connection(**db_creds_test) do.
  • connection_root_bare and connection_test pass backend="mysql".
  • The connection_by_backend fallback is "postgresql".
• tests/integration/test_cli.py::test_cli_schemas starts python -m datajoint.cli against the MySQL test server. Its subprocess now sets DJ_BACKEND=mysql.
• tests/unit/test_settings.py, tests/unit/test_thread_safe.py: these expect the new default and port, and they isolate DJ_BACKEND/DJ_PORT where they assert defaults.

New tests (no server needed)

• tests/unit/test_connection_backend_hint.py:
  • The hint appears for a failed connection and for a missing driver when the backend is the default.
  • It does not appear when the backend comes from the argument, a config assignment, DJ_BACKEND, or datajoint.json.
  • One test makes a real refused connection through psycopg2 to a closed local port.
• tests/unit/test_settings.py, covering the port:
  • The port follows the backend.
  • backend: "mysql" in datajoint.json without a port gives 3306.
  • A port that was set survives a change of backend.
  • Setting the port to None returns it to the default.
• tests/unit/test_thread_safe.py: Instance(backend="mysql") connects on 3306.

Full suite (testcontainers, local): 1170 passed, 14 skipped.

Checked against real servers, with this branch merged with the drivers-as-extras PR and installed in clean virtual environments (no datajoint.json):

Install	Setting	Result
[postgres]	none	Connects to PostgreSQL with the default backend; pymysql not imported
[mysql]	DJ_BACKEND=mysql	Connects to MySQL; psycopg2 not imported
[mysql]	none	Missing-psycopg2 error naming datajoint[postgres], plus the hint
[postgres]	none, pointed at a MySQL server	psycopg2 protocol error, plus the hint

Migration for MySQL users

Use any one of:

• export DJ_BACKEND=mysql
• datajoint.json: {"database": {"backend": "mysql", "host": "..."}}
• dj.config["database.backend"] = "mysql" before connecting
• dj.Instance(host=..., user=..., password=..., backend="mysql")

The port then defaults to 3306, and an explicit port is kept.

Notes for review

• Re-raising as DataJointError changes the exception type only when the backend is the default. Code that catches a driver exception (for example pymysql.err.OperationalError) around dj.conn() sees it unchanged once the backend is set.
• This PR is independent of the drivers-as-extras PR. Together, a MySQL user who has only pymysql installed and sets no backend gets the missing-psycopg2 message plus the hint above.

[MilagrosMarin]

Thanks @dimitri-yatsenko, the default-backend hint is a nice touch for the migration, and keying it off model_fields_set is a neat way to tell "chosen" from "defaulted".

Verified locally:
✅ Unit suite passes on this branch, and on master + #1628 + #1629 merged together
✅ An unset port follows the backend both ways; port=None returns it to the backend default; Instance(backend="mysql") and dj.conn() pick up the backend from config

One issue:

• config.override() restores values with setattr, which marks them as explicitly set. After

  with dj.config.override(database__port=1234): ...
  dj.config["database.backend"] = "mysql"

  the port stays 5432 (now in model_fields_set), so it no longer follows the backend. The same restore marks backend as set after override(database__backend=...), which turns the hint off for the rest of the session. Saving and restoring each field's model_fields_set membership in override() would fix both.

Minor:

• With the default backend, unrelated connect failures (e.g. a wrong password on a real PostgreSQL server) also get the MySQL hint. Fine by me given the migration, just noting it.
• CI ran before #1596 merged, so the py3.15 legs haven't run here. A rebase/re-run would confirm.
• On the docs side, configure-database.md, reference/configuration.md, tutorials/index.md, and installation.md still show 3306 / MySQL as the default. I'll fold those into the docs PR for #1628.

Let me know what you think!