Skip to content

fix: strip <answer> XML tags before do/finish markers in _parse_response - #414

Open
nankingjing wants to merge 2 commits into
zai-org:mainfrom
nankingjing:fix-parse-response-xml-order
Open

nankingjing wants to merge 2 commits into
zai-org:mainfrom
nankingjing:fix-parse-response-xml-order

Conversation

@nankingjing

Copy link
Copy Markdown

Summary

Fixes an ordering bug in ModelClient._parse_response (phone_agent/model/client.py) that corrupts every action when the model answers in the tagged format requested by the system prompt.

The bug

The system prompt (phone_agent/config/prompts_zh.py / prompts_en.py) instructs the model to output:

<think>{think}</think>
<answer>{action}</answer>

where {action} is a do(...) or finish(...) call. _parse_response checked its rules in this order:

  1. finish(message= marker
  2. do(action= marker
  3. <answer> XML tags (documented as the fallback)

Because a tagged answer always contains a do(action= or finish(message= call inside the <answer> block, Rule 1/2 match before the tags are stripped. For input:

<think>I should tap the search box</think><answer>do(action="Tap", element=[500, 300])</answer>

the old code returned:

  • thinking = '<think>I should tap the search box</think><answer>' (tags leaked in)
  • action = 'do(action="Tap", element=[500, 300])</answer>' (trailing </answer>)

The action string is not valid Python, so the downstream parse_action (phone_agent/actions/handler.py) raises ValueError. In PhoneAgent._execute_step that exception is swallowed and turned into finish(message=<raw string>), so the agent ends the task immediately instead of tapping. As a consequence, Rule 3 (the XML-tag path) was effectively dead code — it could never be reached for a real action.

The fix

Check for the <answer> tag first. When present, strip <think>/</think>/<answer>/</answer> and return the clean thinking/action pair. Tag-less responses (the AutoGLM-Phone-9B happy path) are unchanged: they fall through to the unchanged finish(message=/do(action= marker rules.

This is a pure reordering — no tag-less behaviour changes.

Tests

Adds tests/test_parse_response_order.py:

  • Tagged do/finish responses now yield a clean action with no </answer> remnant (regression cases).
  • Tag-less do, finish, finish-precedes-do, and plain-text fallback still behave exactly as before.

Verification

Not executed in CI here. Verified by reading the code path end-to-end and by simulating the reordered _parse_response against all test inputs (all produce the asserted (thinking, action) tuples). Both edited files pass ast.parse.

@nankingjing

Copy link
Copy Markdown
Author

Reviewed — moving the tag check before the marker checks correctly prevents trailing from leaking into the action string. Good test coverage. Ready for review.

@nankingjing

Copy link
Copy Markdown
Author

#378 and this PR fix the same bug: the <answer> tag check sits after the do(action= / finish(message= marker checks, so a tagged response leaves a trailing </answer> inside the action string, which then fails parse_action and aborts the step. Only one should merge, so here are the concrete differences.

This PR

  • Reorders the three rules only. ModelClient._parse_response keeps its name and signature, so no caller breaks. +24/-14, no unrelated edits.
  • Strips <think>/</think> from everything before <answer>, so the thinking comes back tag-free whether or not the closing tag is present.
  • 104 lines of tests, 7 cases, covering the tagged path and the tag-less paths it must not regress.

#378

  • Adds a debug print to the request path at phone_agent/model/client.py:147:

    print(f"{thinking=}, {action=}")

    That writes every parsed model response to stdout on every step.

  • Moves _parse_response off the class to a module-level parse_response. Any caller using client._parse_response(...) breaks.

  • Its rule 1 requires both tags (if "<answer>" in content and "</answer>" in content). With only the opening tag present it falls through to the do(action= marker, which leaves the raw tags in the thinking string — and its own last test pins that as the expected result:

    content = """<think>先思考</think>
    <answer>
    do(action="Launch", app="知乎")"""
    thinking, action = parse_response(content)
    assert thinking == "<think>先思考</think>\n<answer>"   # tags leak in here

    A truncated model response is exactly this shape, so that assertion locks the leak in rather than fixing it. This PR returns 先思考 for the same input.

One thing #378 does better: it truncates the action at the first closing tag (parts[1].split("</answer>", 1)[0]) instead of deleting all of them, which drops any trailing text after </answer>. I'll fold that into rule 1 here if there's interest.

I've left a note on #378 rather than closing it, since it isn't mine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant