Skip to content

feat: add owned parallel task collaboration and selective cancellation - #7643

Open
me-her wants to merge 5 commits into
google:mainfrom
me-her:codex/agent-worker-concurrency
Open

me-her wants to merge 5 commits into
google:mainfrom
me-her:codex/agent-worker-concurrency

Conversation

@me-her

@me-her me-her commented Oct 21, 2026 •

Copy link
Copy Markdown

Link to Issue or Description of Change

Closes #7641. Related: #7642.

A workflow coordinator can fan out work today, but it lacks an owned group of
handles for inspecting, messaging, selectively cancelling, and accepting results
from those concurrent children. Agent workers also lack the concurrency limit
already available to function workers.

Add Context.parallel_tasks() and public ParallelTask/ParallelTasks types,
using the existing dynamic node scheduler and Runner event stream:

@node(rerun_on_resume=True)
async def coordinate(ctx: Context, node_input: str):
  async with ctx.parallel_tasks(name="research", max_parallel_workers=3) as tasks:
    handles = [tasks.spawn(agent, node_input) for agent in researchers]
    handles[2].send_message("Check the counterexamples as well.")
    winner = await tasks.first(accept=verify_answer)
    await tasks.merge_result(winner, keys=["answer", "sources"])
    return winner.output

The coordinator can continue while children run, wait for any/all handles, gather
ordered results, or cancel an individual owned task. first evaluates synchronous
or asynchronous acceptance predicates, records its winner, and cancels and drains
remaining work before returning. Rejected results and failed candidates allow
alternatives to continue. Queued admission pauses while acceptance is evaluated.

Explicit task groups add correlation to native events. Bounded FIFO messages have explicit queued,
delivered, and undelivered receipts; delivery occurs at the addressed agent's next
model boundary. Cancellation preserves completed sibling tool results and closes
otherwise unanswered local tool calls. Durable cancellation and merge decisions
survive human-input resume, including transferred child runs.
Completed siblings retain their results through native replay. A restored winner
cannot be blocked by an unnecessary retry occupying the concurrency limit, and
retrying selection after a cleanup timeout still waits for actual termination.

Ordinary parallel_worker=True uses the same task group while retaining ordered
outputs, event metadata, fail-fast cleanup, resume, and O(concurrency-limit) active handles. Expose
max_parallel_workers on Agent/LlmAgent and preserve the original agent when
constructing its internal worker copy. Update two GEPA mocked-agent fixtures for
the new default field.

State remains native shared session state. Explicit merge accepts returned mapping
proposals, validates selected keys before writes, and records the state delta and
merge disposition in one event. It does not isolate arbitrary worker writes or
undo external side effects. Cancellation is cooperative; a cleanup timeout is
reported, and already sent provider requests may still incur cost. This change
makes no Rust or production latency claim.

The usage guide and runnable three-agent verified-winner sample are included in
this PR. Test fixtures initialize event metadata and consistently
mock cached credential aliases, keeping the full suite independent of local ADC.

Testing Plan

The current PR revision is 0bc6a28c82a59bbec2101af79180d56479095b86, which
merges three upstream fixes into the previously validated implementation.
The collaboration runtime and its tests are unchanged; the merge preserves the
message-delivery hook in the updated model flow.

The current-revision Linux validation run
passes the full ADK unit suite using uv on Python 3.11–3.14:
20,780 / 20,771 / 20,771 / 20,772 passed, respectively, with no failures or
errors. Runs retain 93–94 skips, 18 expected failures, and two non-strict
unexpected passes. Checkout logs confirm the full revision above in all four
jobs. This is an independent validation run; Google's upstream CI still awaits
maintainer approval.

Earlier implementation validation targets
d36495faae14b7514bf05e9d41fe32fdf5e1298c (before the upstream merge):

  • Deterministic Runner tests exercise concurrency, parent progress, rejection,
    cancellation, model/tool cleanup, bounded admission, observer cancellation,
    failures, messaging races, merge conflicts, and both native resume modes.
  • The combined new and existing worker, replay, and tool regression selection
    passes: 407 tests on uv-managed Python 3.12.
  • The earlier full Linux matrix
    also passes on Python 3.11–3.14, with over 20,700 tests per version.
  • Installed-wheel smoke scripts exercise
    native collaboration
    and agent-worker limits
    with scripted models and no provider credentials. The collaboration case verifies
    four simultaneous agent calls, a rejected early result, an accepted winner,
    delivered steering, closed losing generators, progress correlation, and persisted
    selected state. Worker caps 1, 2, and unlimited produce peaks 1, 2, and 4 with
    ordered outputs.
  • All applicable pre-commit hooks pass on every changed file. uv build produces
    the wheel and source distribution.
  • Strict mypy on all 13 changed source files reports four existing errors;
    comparison with unchanged upstream 7c53386 introduces no new errors.
  • The offline selection benchmark
    records 12 repetitions per policy against the d36495f installed wheel:
    median wall time 138.4 ms for gathering all answers versus 80.6 ms for the
    first acceptable result. Both start four local model calls; the latter
    cancels two workers and verifies all generators close. Fixed simulated
    delays establish selection behavior, not production latency, token savings,
    or a Python/Rust comparison.

Checklist

  • I have read CONTRIBUTING.md.
  • I have added regression tests and a usage guide/sample.
  • Source review and focused validation are complete for the implementation.
  • The upstream merge preserves the collaboration implementation and tests.
  • Full unit-suite results have been recorded for the current merged revision.
  • Installed-wheel checks passed for implementation revision d36495f.

Expose max_parallel_workers on Agent and forward it to the existing worker scheduler. Validate worker-only limits and preserve ordered outputs, cancellation, and resume behavior.

Fixes google#7641
@me-her me-her changed the title feat(workflow): support bounded parallel agent workers feat: add owned parallel task collaboration and selective cancellation Oct 21, 2026
Serialize cancellation persistence and terminal status. Retain completed sibling identities through native replay, free saved-winner capacity from unneeded retries, and require cleanup before repeated winner selection succeeds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support max_parallel_workers on Agent parallel workers

2 participants