📖
ToolRosetta: Surface-Form Invariance of Tool-Call Guards
Hasan Demirkiran, Jens Ernstberger

ToolRosetta: Surface-Form Invariance of Tool-Call Guards

Author / Organization:
Published 2026 Verification Pending

Overview & Synopsis

Step-level guards for tool-using agents are evaluated on tidy JSON tool calls, but deployed coding agents act through execute_bash and execute_code tools whose payloads are strings. We introduce ToolRosetta, which re-renders every candidate action in the AgentDojo and AgentHarm splits of TS-Bench as a Python call, as a shell command, and without the agent's rationale. The renderings are produced by code and verified by round-trip parsing, so the user request, the history, the label and the action's effect stay fixed while only the surface form changes. Across eleven guards with two placebo controls, frontier and strong open-weight guards are stable: no frontier error rate moves by more than 3.4 points. Four of five specialized agent guards are not: under the shell form their false-positive rates rise by 21 to 47 points on AgentDojo, and TS-Guard's by 38 points on its own home benchmark, blocking between a third and nine in ten benign actions; the fifth, StepGuard, does not move. A small open guard drifts five points more permissive. Removing the agent's rationale moves most guards whose input carries it, in guard-specific directions. A request-blind control on one guard shows that the rationale is where intent is read from. A parser that restores the canonical call in front of the guard removes the effect for these renderings, by construction. Benchmark accuracy on JSON calls does not predict a guard's operating point behind a code-acting agent.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title ToolRosetta: Surface-Form Invariance of Tool-Call Guards
Author / Organization Hasan Demirkiran, Jens Ernstberger
Subject & Discipline cs.CR
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-278662
Original Source Link Visit Source Page on arxiv.org »