[Bug][Structured Outputs] Parser._apply_structural_tag overrides a request's response_format when tools are present — schema not enforced (GLM-5.3) #62483

Environment

• vLLM: nightly 77154ca61 (0.32.0rc1.dev100) and 39983a5e7 (0.32.0rc1.dev329) — both reproduce
• Model: GLM-5.3-Flash (reasoning model; reasoning parser glm45, tool parser glm47, tools supplied in the request)
• Structured outputs: xgrammar 0.2.8, backend auto
• Speculative decoding: DFlash2 probabilistic, k=7 (the repro does not depend on spec — it fires with spec on; we have not isolated spec-off)
• Hardware: 2× NVIDIA DGX Spark (GB10), TP=2 — likely irrelevant
• The failure is deterministic, single-turn, and does not involve the reasoning parser path

Observed failure

A request carrying both tools and response_format (json_schema) does not enforce the schema. The model free-flows into tool calls on the schema'd turn:

finish_reason = "tool_calls", content = null
tool_calls = [{"function": {"name": "calculator", "arguments": "{\"expression\": \"28\"}"}}]

while the same request without the tools array enforces the schema correctly (a const-forcing schema produces exactly the forced string, finish_reason = "stop"). Deterministic on every attempt.

In multi-turn agentic workloads this compounds: the model re-issues tool calls on schema'd turns, the tool-call structure never completes under the composed grammar, and generation runs until the token budget is exhausted (we observed a single request produce ~130K tokens across repeated 4096-token turns).

Root cause

Parser._apply_structural_tag (vllm/parser/abstract_parser.py, ~lines 505-603 at 77154ca61): when the tool parser supports structural tags and the request carries tools, it composes

OrFormat([tool_call_tag, output_format])

from the request's own response_format, assigns it to request.structured_outputs, and nulls request.response_format. Under the Or, the tool-call branch is legal — so on a schema'd turn the model's preferred continuation (another tool call) passes the grammar and the user's schema is never enforced.

Why this is new (and why 0.31.0 was unaffected)

The composition code exists in 0.31.0 (db9527a468) as well, but it never armed for GLM: structural_tag_registry had no glm_4_7 builder and xgrammar 0.2.7 lacked the builtin, so tools_structural_tag was None and the function returned early with response_format intact — schema-only grammar. The 0.32 nightly added the _glm_4_7_tool_tag builder (abstract_parser.py:1047) together with xgrammar 0.2.8, which armed the composition for GLM.

Minimal reproduction

Server: any GLM-5.3 checkpoint served with --enable-auto-tool-choice --tool-call-parser glm47. Client (single turn, temperature 0):

import json, urllib.request

schema = {
    "type": "object",
    "properties": {"answer": {"type": "string", "const": "GRAMMAR_WORKS"}},
    "required": ["answer"],
    "additionalProperties": False,
}
tools = [{
    "type": "function",
    "function": {
        "name": "calculator",
        "description": "Evaluate an arithmetic expression.",
        "parameters": {"type": "object",
                       "properties": {"expression": {"type": "string"}},
                       "required": ["expression"]},
    },
}]

def run(with_tools):
    body = {
        "model": "<model>",
        "messages": [{"role": "user", "content": "What is 2+2? Answer with the required field."}],
        "max_tokens": 512, "temperature": 0,
        "response_format": {"type": "json_schema", "json_schema": {"name": "t", "schema": schema}},
    }
    if with_tools:
        body["tools"] = tools
        body["tool_choice"] = "auto"
    req = urllib.request.Request("<base-url>/v1/chat/completions",
                                 data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=180) as r:
        out = json.loads(r.read())
    c = out["choices"][0]
    print("tools =", with_tools,
          "| finish =", c["finish_reason"],
          "| content =", repr(c["message"]["content"]),
          "| tool_calls =", repr(c["message"]["tool_calls"])[:80])

run(with_tools=False)  # schema enforced: content == '{"answer": "GRAMMAR_WORKS"}', finish=stop
run(with_tools=True)   # schema ignored:  finish=tool_calls, content=None

Related issues

• [RFC]: Strcutured Outputs Not Respected When Reasoning Implicitly Ends #60722 (RFC): implicit reasoning ends break structured outputs + spec decode. The Or's tool-call branch is precisely the implicit-end escape hatch this RFC analyzes; our failure is the sampling-half consequence on GLM-5.3.
• [Bug]: Intermittent XGrammar request termination with V2 DFlash probabilistic drafting and block rejection #61383: a sibling failure mode of the same spec×grammar coexistence (grammar-invalid committed tokens → fail-closed termination), where the fail-closed tripwire catches what our case slips past.

Workaround (validated)

With tool_choice=auto, keep the user's schema as the only grammar — do not compose the tool-call branch (this restores the 0.31.0 behavior). required/named tool_choice still compose, since a forced call is the user's explicit intent. With this change, all four previously-failing structured-output scenarios in our eval suite recover to full score, and eval parity with 0.31.0 is restored.

Question for maintainers

Should tool_choice=auto widen the grammar at all when the user explicitly requested a response_format schema? If composition stays for auto, should the tool branch be gated per the strict-thinking direction proposed in #60722?