[gonzalo-zenrows]

Why

A request that fails in a way another try can fix should wait and try again. It should not fail at once, and it should not retry on a fixed schedule. For example, a cold Extract request can briefly answer 503 with "retry in a minute, not charged". Today ZenRowsClient retries nothing by default. When a caller turns retries on, every client waits the same fixed time, so many clients retry together.

What this changes

ZenRowsClient now retries a failed request up to 2 times by default. It retries only failures that another try can fix, and only requests that are safe to send again.

The client retries these failures:

1. A 429, 500, 502, 503 or 504 response.
2. A 422 response, except Extract's no_data_extracted. That one is a final answer.
3. A failure to connect: a refused connection, a name that did not resolve, or a connect timeout. The request never reached the server.

The client never retries:

1. Any other status, such as 400, 401, 402, 403 or 404.
2. A read timeout.
3. A connection that closes after the client sent the request. The API may already be working on that request.
4. Any status when the call sets original_status to true. The API then returns the target site's own status, and the client cannot tell it from the API's.
5. A status listed in the call's allowed_status_codes. The API passes that status through from the target site, so another try cannot change it.

The client reads both parameters from each request's query string. That covers True, "true", 1 and "1" for original_status, in any case. For allowed_status_codes it takes a comma-separated string or a list. Both the urllib3 retry and the 422 retry check them. A failure to connect is not a status, so the client still retries it.

The client retries these requests:

1. A GET request whose body is empty, a string, bytes, or a dict the client encodes.
2. The client never retries a POST or PUT. It sends each one once, whatever its headers, its body, the status or the failure. That includes a refused connection.
3. An Idempotency-Key header changes nothing. The API does not deduplicate Fetch or Extract requests by that key, so a second try could run the request twice.
4. The client never retries a request whose body is a file object, a generator, an iterator or another stream. The first try reads that body, so a second try would send it empty.
5. A caller who needs to retry a POST writes that retry in their own code.

This also fixes a bug in the 422 retry. Before this fix, a POST with an Idempotency-Key and an io.BytesIO body got a 422. The retry then sent the body again, empty, and the server answered 200. Now the client sends the file body once and returns the 422 as it came. The 422 loop and the urllib3 retry each check the body they would send again.

Before each retry, the client waits:

1. The Retry-After the API sent, when it is a whole number of seconds. The client clamps it to between 0.25 s and 30 s. The 30 s cap is fixed and does not follow retry_max_wait.
2. A Retry-After of any length never raises. The client gives a value above 99 the 30 s cap and does not parse it.
3. Otherwise, a random time between 0.25 s and max(0.25, min(retry_max_wait, 2 ** attempt)) seconds. Here attempt is 0 for the first retry. An HTTP date, a fraction or any other Retry-After value also gets this random wait.

No retry starts after 90 s. The client does not start a retry when the time since the first try plus the next wait would pass the budget. The budget does not cut a running try short, so a call can run past it by one try.

A 422 retry runs in _worker, because urllib3 cannot read the response body. It uses the same wait, the same retry count and the same budget clock as the urllib3 retries. A 422 and a 5xx in one call therefore share one count and one budget. After the last try, the client returns the final response as it came.

prepare_domain() and prepared_domain_status() never retry. Their 429 and 503 are answers for the caller to read, so the client sends these calls once.

The client checks its options when you create it. retries must be an integer of 0 or more. retry_max_wait and retry_budget must be finite numbers above 0. Anything else raises ValueError.

A call cannot be cancelled. A wait between tries runs to its end, also for the *_async methods, which run the call on the client's thread pool. The README says so.

Parameter	Before	Now
retries	0 (off)	2
retry_max_wait (new, keyword-only)	n/a	30.0 seconds: the cap on the random wait
retry_budget (new, keyword-only)	n/a	90.0 seconds: no retry starts after this

To keep the old behaviour of no retries, pass retries=0.

Behaviour change

Callers who upgrade will see these differences:

1. A client that sets no retry count now retries up to 2 times instead of never.
2. The client never retries a POST or PUT. On main, a client with retries set retried a PUT that got a 503. That PUT is no longer retried.
3. On main, such a client also retried a POST or PUT after a 422 or a refused connection. It no longer does.
4. The client no longer retries a GET after a read timeout or a connection that closes after the request was sent. On main, a client with retries set retried both.
5. The client no longer retries a request whose body is a file object, a generator or an iterator. It also never retries a 422 for a request that uploads files: the second try would send the file part empty.
6. The waits are now random, honour Retry-After, and no retry starts after 90 s.
7. An invalid option now raises ValueError when you create the client. Examples are retries=2.5, which main accepted, and retry_max_wait=-1.
8. A call that sets original_status gets no status retry. A status listed in allowed_status_codes is not retried. On main, a client with retries set retried both.

The README has a "Retries" section that names the changed default and says that POST and PUT are never retried.

Tests

tests/test_retries.py runs the client against a local HTTP server, so each test drives urllib3's real retry path. It covers:

1. Every Retry-After and jitter vector of the shared retry policy, as parametrised tests.
2. Retry on 429, 500, 502, 503, 504, a 422 other than no_data_extracted, and a refused connection.
3. No retry on 400, 401, 402, 403, 404, 422 no_data_extracted or 413 with Retry-After. No retry on a read timeout. No retry on a connection that closes after the send (GET, POST and PUT).
4. A POST or PUT sent once, with and without an Idempotency-Key, for a 503, 429, 422, 500 and a refused connection. The async post_async and put_async are sent once too.
5. The io.BytesIO repro: a POST with a key and a file body gets a 422 or a 503. The client sends the body once and returns that status.
6. A GET with files gets a 422: the client sends the file once and returns the 422. On a 503 it sends the file in both tries.
7. A GET with a file, generator or iterator body sent once on a 503 or 422. A GET with a string, bytes or dict body sent again with the same body.
8. One count, one attempt index and one budget for a 422 and a 5xx in the same call.
9. One count and one budget per call on fetch_async, which runs on the client's thread pool.
10. extract() returning a 403 in one request, with no retry.
11. prepare_domain() and prepared_domain_status() sending a 429 or 503 once, sync and async, with no wait.
12. Option validation.
13. A call with original_status set to True, "true", "TRUE", 1 or "1" that gets a 429, 500, 502, 503, 504 or 422. The client sends it once.
14. A call with original_status absent, False, "false", 0, "0" or empty that gets a 503 or 422. The client still retries it.
15. A call with allowed_status_codes that lists 503 gets a 503. The client sends it once, for a string, a spaced string or a list. A 502 or 422 not in the list is still retried.
16. A 422 listed in allowed_status_codes, and extract() with original_status, each sent once. This drives the 422 loop.
17. A refused connection with original_status set: the client still retries it.
18. One call's original_status does not stop the next call's retries.
19. A 429 with a 5000-digit Retry-After. The client waits 30 s once and returns the next response.

Results on this branch:

1. uv lock --check and make check: passed.
2. make test: 498 passed on urllib3 2.8.0 with Python 3.10.
3. The full suite also passes on urllib3 1.26.20: 498 passed.
4. A mutation pass reverted each part of the POST and PUT change one at a time. Each of these eight mutations made a test fail:
   1. Retry every method.
   2. Retry every method in the urllib3 path only.
   3. Retry every method in the 422 loop only.
   4. Drop the body check in the 422 loop.
   5. Drop the body check in the urllib3 path.
   6. Bring back the Idempotency-Key rule.
   7. Let urllib3 ignore the per-request check.
   8. Drop the whole check in the 422 loop.
5. A second mutation pass covered the merge of main. Each of these four mutations made a test fail:
   1. Send prepared_domain_status() through the retry session.
   2. Send a second request after a 403 in extract().
   3. Keep one call's retry state alive for the next call.
   4. Give the 422 loop and urllib3 separate retry state.
6. A third mutation pass covered the target-site rule, on urllib3 2.8.0 and 1.26.20. Each of these twelve mutations made a test fail:
   1. Drop the check in the urllib3 retry.
   2. Drop the check in the 422 loop.
   3. Stop splitting allowed_status_codes on commas.
   4. Stop trimming spaces around each code.
   5. Read only the first of a repeated allowed_status_codes key.
   6. Ignore allowed_status_codes.
   7. Ignore original_status.
   8. Read original_status case-sensitively.
   9. Stop reading 1 as true.
   10. Read any original_status value as true.
   11. Stop the adapter from setting the per-request check.
   12. Let original_status also stop a retry after a refused connection.
7. A fourth mutation pass covered the long Retry-After fix. Each of these two mutations made a test fail:
   1. Parse the whole value with int() again.
   2. Apply the length rule before the client strips leading zeros.

Not covered

1. The client has no cancel signal, so a caller cannot stop a wait between tries. The README documents this; the change does not add one.
2. The client does not retry a POST or PUT. If the API starts to deduplicate by Idempotency-Key, a new decision can add that retry back.
3. The Batch and Crawl clients keep their own retry transport. This change does not touch them.
4. The public SDK pages on docs.zenrows.com still describe the old defaults. They live outside this repository.
5. allowed_status_codes tokens that are not whole numbers are ignored. The client does not read ranges such as 4xx.
6. The release note is the text under "Release note" below. The PR title stays a plain feat, so the version bump is a minor one.

Release note

Paste this text into the release PR body when it opens, and check the version number.

Retries are now on by default.

1. ZenRowsClient retries a failed request up to 2 times. Before, the default was 0. Pass retries=0 to keep the old behaviour.
2. It retries a GET that gets a 429, 502, 503, 504, 500, or a 422 other than no_data_extracted. It also retries a connection that failed before the request left: refused, name not found, connect timeout.
3. It never retries a POST or PUT, a read timeout, a connection that closed after the request was sent, or a status that comes from the target site (when original_status is set, or the status is in allowed_status_codes). A request whose body is a file or a stream is not retried either.
4. Before each retry it waits the Retry-After the API sends, when that is whole seconds (0.25 to 30 seconds). Otherwise it waits a random time between 0.25 seconds and a ceiling that starts at 1 second and doubles.
5. No retry starts after 90 seconds from the first try. A call can run longer than that by one attempt.

What you may notice when you upgrade

1. A call that failed at once on a 429 or 503 can now wait up to about 90 seconds before it fails.
2. A PUT is no longer retried. The old client retried it on a 503 when retries was set.
3. New options retry_max_wait and retry_budget. An invalid retries, retry_max_wait or retry_budget now raises ValueError when you create the client.

[gonzalo-zenrows]

Same retry policy across all clients (429/502/503/504 and pre-response connection errors; 2 retries by default; full jitter; Retry-After capped at 30 s; 90 s total budget; timeouts never retried):

• Python SDK: feat(client): retry only failures another try can fix, with jittered backoff #62
• Go SDK: feat(api): retry transient failures by default with full-jitter backoff zenrows-go-sdk#43
• Node SDK: feat(client): retry uncharged failures by default, with full-jitter backoff and a time budget zenrows-node-sdk#58
• CLI: feat(fetch): retry uncharged busy answers so a brief capacity wait does not fail the command cli#64
• MCP: feat(fetch): retry not-served scrape and extract calls so a busy API does not fail the tool call zenrows-mcp#80

[jonathan-zenrows]

Two bugs, plus one thing to decide before merging.

1. The 422 retry sends an empty body when data is a file object

_can_send_again accepts a seekable file, and the 5xx retries rewind it (urllib3 handles body_pos). The 422 loop in _worker (client.py:642-647) calls send() again with the same data, which is now at EOF. The second request goes out with an empty body and the caller gets a 200.

Repro: run a local server that answers 422 and then 200, and log each request body:

cl = ZenRowsClient("k", base_url=base)
cl.post("http://t", headers={"Idempotency-Key": "a"}, data=io.BytesIO(b"PAYLOAD"))
# bodies seen by the server: [b'PAYLOAD', b'']  -> status 200
# same thing with 503 then 200:  [b'PAYLOAD', b'PAYLOAD']

Fix: save data.tell() before the first send and seek() back to it before each 422 retry. Or don't retry 422 for anything that isn't str, bytes or dict. The file-body test in test_retries.py:398-416 only runs with 503; add 422 to its parameters.

2. A long Retry-After raises

int(value) in _retry_after_seconds (client.py:74) raises ValueError on 3.11+ when the value has more than 4300 digits. The error reaches the caller unwrapped. Repro: a 429 with Retry-After: "9" * 5000. Fix: if len(value) > 2, return the cap without parsing; anything past 2 digits is above 30 s anyway.

3. Release note

The default goes from 0 to 2 retries, PUT now needs an Idempotency-Key, GET read errors are no longer retried, and retries=2.5 now raises. With feat(client): and skip-changelog: true, this ships as a plain minor and the title is the only note. It needs ! or a BREAKING CHANGE: footer, or at least a written note on the release.

Also: this conflicts with main (1.11.0 was released this morning), so it needs a rebase.

Rest of the spec checked and fine. The suite passes locally (393).