Repository navigation
sdk/python: refuse Python 3.14 with a clear error, prefer 3.13 to 3.11 - #709
asivanadi0 wants to merge 3 commits into
Conversation
py-libp2p pins coincurve 21.0.0, which ships no wheels past CPython 3.13 (libp2p/py-libp2p#1140), so pip install on 3.14 fails deep in a native build. Cap requires-python at <3.14, have make sdk-python pick the newest 3.11 to 3.13 interpreter on PATH (probed once, into a simply expanded variable), and stop with a clear message when PYTHON is out of range. An explicit PYTHON= still wins. Fixes google#692 Signed-off-by: asivanadi0 <sivanadi08@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request restricts the Python SDK to Python versions 3.11 through 3.13 due to a dependency limitation with py-libp2p pinning coincurve 21.0.0. It updates pyproject.toml and modifies the Makefile to automatically detect and validate a compatible Python interpreter. Feedback is provided to optimize the Makefile by making the SDK_PYTHON_AUTO variable evaluation lazy, preventing unnecessary shell executions and overhead during unrelated make invocations.
| # PYTHON defaults to the newest such interpreter on PATH, probed once when | ||
| # the Makefile is read; override it with make sdk-python PYTHON=/path/to/python3.12. | ||
| SDK_PYTHON_CHECK := import sys; sys.exit(not (3, 11) <= sys.version_info[:2] < (3, 14)) | ||
| SDK_PYTHON_AUTO := $(or $(firstword $(foreach p,python3.13 python3.12 python3.11 python3,$(shell $(p) -c '$(SDK_PYTHON_CHECK)' >/dev/null 2>&1 && echo $(p)))),python3) |
There was a problem hiding this comment.
Using a simply expanded variable (:=) for SDK_PYTHON_AUTO causes the shell commands to be executed on every invocation of make (e.g., make build, make clean), even when the sdk-python target is not being run. This adds unnecessary overhead and latency to all build commands.
We can make this lazy (evaluated only when needed, and only once) by using a recursively expanded variable with eval to redefine itself on first use.
SDK_PYTHON_AUTO = $(eval SDK_PYTHON_AUTO := $(or $(firstword $(foreach p,python3.13 python3.12 python3.11 python3,$(shell $(p) -c '$(SDK_PYTHON_CHECK)' >/dev/null 2>&1 && echo $(p)))),python3))$(SDK_PYTHON_AUTO)
There was a problem hiding this comment.
@asivanadi0 You can improve this makefile commands here as suggested by bot.
There was a problem hiding this comment.
Thanks @yashksaini-coder, done in f32c836. SDK_PYTHON_AUTO now uses Gemini's lazy eval form, so the interpreter probe runs only when sdk-python needs it, and only once, instead of on every make invocation. Behavior is unchanged: it still prefers python3.13 down to 3.11 and refuses 3.14 with the same error. make sdk-python passes locally on 3.13.
|
I'm not fully familiar with the python packaging ecosystem these days but is the common practice to do this kind of workaround? |
|
Good question. Capping requires-python is generally discouraged for published libraries, since it can make resolvers fall back to older releases, so I only did it here because the install currently fails outright on 3.14 (coincurve 21.0.0 has no 3.14 wheels, libp2p/py-libp2p#1140). If you'd prefer, I can drop the pyproject cap and keep only the Makefile check, which only affects make sdk-python and gives a clear error instead of a native build failure. |
Make SDK_PYTHON_AUTO a recursively expanded variable that redefines itself via eval on first use, as suggested in review, so other make targets no longer shell out to probe python3.13 to python3.11 on every invocation. Signed-off-by: Anpoo Sivanadi <sivanadi08@gmail.com>
yashksaini-coder
left a comment
There was a problem hiding this comment.
Retested this on the Python 3.14 host from #692 (now 3.14.8, with no python3.11/12/13 on
PATH at all), against your branch rather than a reconstruction. It behaves correctly in every
case I could think to try:
| 3.14-only host, no override | guard fires, exit 1, and no venv is created — it stops before touching anything |
python3.13 also on PATH |
selects python3.13 |
PYTHON= on the command line |
honoured |
PYTHON= from the environment |
honoured |
And the full target on the auto-selected interpreter:
python3.13 -m venv sdk/python/.venv
167 passed in 50.09s # sdk/python/tests
35 passed in 11.75s # development/tools/agentmesh-cop
Your memoisation is better than what I suggested, and worth calling out for anyone reading this
later. I proposed a simply expanded SDK_PYTHON_AUTO :=, which caches but evaluates eagerly —
the probes would then run on every make invocation, make build included. You used
SDK_PYTHON_AUTO = $(eval SDK_PYTHON_AUTO := $(or ...))$(SDK_PYTHON_AUTO)which is lazy and run-once, so it resolves the trade-off I left open. Measured with a counting
shim on PATH:
make build→ 0 probesmake sdk-python→ 1 probe per candidate
Nothing blocking from me. LGTM.
One piece of context rather than a request, since it is not this PR's job to fix: the
requires-python cap is enforced by pip, so make sdk-python and pip install agentmesh-p2p
both get a clear message, but uv pip install -e ./sdk/python does not honour it for a local
editable path — it resolves and still fails in coincurve's build. Your make guard covers the
repo workflow, which is the case that matters here; I mention it only so nobody later assumes the
cap alone protects every installer.
|
I do not understand why we should workaround the problem here, at least users will know it will not work ... |
agentmesh-p2p is already published with releases that declare only ">=3.11", so capping the next release does not stop a 3.14 install: pip silently falls back to an older uncapped release, which then fails the same way on coincurve's missing 3.14 wheels (libp2p/py-libp2p#1140). Restore requires-python to ">=3.11" and keep the 3.11 to 3.13 check in make sdk-python, which gives a clear error when building from the repo. Signed-off-by: Anpoo Sivanadi <sivanadi08@gmail.com>
|
Thanks @yashksaini-coder for the pip demo on #692. A capped release just makes pip on 3.14 fall back to 0.1.0, which fails on the same coincurve build, so the cap buys nothing. @aojea agreed, the cap is dropped in d291892: |
py-libp2p pins coincurve 21.0.0, which ships no wheels past CPython 3.13 (libp2p/py-libp2p#1140), so
make sdk-pythonon a 3.14 host fails deep inside coincurve's native build with an error that doesn't mention the Python version.This PR:
It leaves
requires-pythonin sdk/python/pyproject.toml at>=3.11. An earlier revision capped it at<3.14, but as @yashksaini-coder showed on #692, a cap on a new release only makes pip on 3.14 fall back to an older uncapped release, which fails the same way.Thanks to @yashksaini-coder for reporting this, testing the branch on a Python 3.14 machine, suggesting the run-once probe, and the pip downgrade demo.
Fixes #692