Repository navigation
Release GIL during image encoding and map loading - #301
Open
JabberWocky-22 wants to merge 2 commits into
Open
JabberWocky-22 wants to merge 2 commits into
JabberWocky-22 wants to merge 2 commits into
Conversation
Generated with Codex.
Release the GIL around load_map and load_map_from_string using a pybind11 call guard. Add a parametrized regression test for both loading paths and document the fix in the changelog. Generated with Codex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #300.
Formatted
Image.to_string()andImageView.to_string()encoding currently hold the GIL, so encoding in a Python worker thread still blocks other Python threads.Release the GIL only around the native
save_to_string()calls in all four formatted overloads, including explicit-palette variants. The GIL is reacquired before constructing Pythonbytes, including on the exception path. Encoding algorithms and parameters are unchanged.Add a parametrized regression test covering Image/ImageView with and without a palette. It checks that another Python thread can run during encoding and that decoded pixels are preserved, without latency thresholds. Also add a changelog entry.
Validation
Built the native Mapnik v4.3.2 core and original/patched Python bindings in an isolated Docker environment:
image_test.py,png_encoding_test.py,palette_test.py,image_tiff_test.py, andwebp_encoding_test.py.No C++ coverage report was collected.
Additional fix: map loading
load_map()andload_map_from_string()have the same GIL retention problem. Datasource initialization during loading can perform database I/O, so loading in a worker thread may still block other Python threads while waiting for a database.Add
py::call_guard<py::gil_scoped_release>()to both bindings. Argument conversion happens before the GIL is released, and the GIL is reacquired when the native call returns or throws. The existing image-encoding commit is retained; this fix is a separate additional commit.Add one parametrized regression test for file and string loading. It checks thread progress during loading and verifies that all 2,000 layers are loaded, without latency thresholds or external database dependencies. Add a changelog entry.
Additional validation in an isolated Docker container:
load_map_test.py, including the two new cases.This additional validation did not rebuild the complete Python extension.
Implementation, tests, and PR text generated with Codex.