Adaptive attackers broke 12 published prompt-injection defenses that had reported near-zero attack success, pushing most above 90%.
AI Agent Prompt Injection Rankings 2026
Claude 3.5 Sonnet resisted all but 1.11% of AgentDojo's standard prompt injections, yet NIST's red team hijacked the same generation of agent 81% of the time with adapted attacks.
- No. 1: Claude Opus 5 (Anthropic)
- 10 places
01Final list
The leading AIs each built their own list of ten independently. Here is one result that is easy to check: every place has a reason, a source, and the list of AIs that included it.
Only systems with reproducible results on both kinds of test, with the model, agent setup and benchmark version named
A bright badge means this AI put the entry on its list. A dim badge means it did not.
- 1
Claude Opus 5 (Anthropic)
Best pairing found: 83.39% on OSWorld-Verified and 0.54% GUI injection success. The injection figure is Anthropic's own; no AgentDojo or ASB run is published.
Included it: Claude, MiMo, Kimi, DeepSeek. Did not include: ChatGPT, Meta, Grok, Gemini, Wenxin, Qwen.In their lists: Claude: 1 · MiMo: 3 · Kimi: see list · DeepSeek: 1
What each AI noted 4
- Claudeplace 1
0.54% attack success in GUI tests, with 70.6% OSWorld 2.0 partial from the same model.
Source ↗ - MiMoplace 3
The most independently cross-checked agent: its numbers survive a rival's run with only a small haircut.
Source ↗ - Kimi
Named differently in its own list: open the full list.
see list - DeepSeekplace 1
85% on OSWorld-Verified and 0% targeted ASR on several AgentDojo suites in the closest reproducible harness—but the harness is not Claude Desktop itself.
Source ↗
4 AIs put it here - 2
OpenAI GPT-5 family (GPT-5.1 / 5.4 / 5.5)
GPT-5.1 is hijacked 4.96% of the time on AgentDyn, about a tenth of GPT-4o's rate, and GPT-5.4 went 0 for 288 in a 2026 replication. GPT-5.5 scores 78.7% on OSWorld-Verified.
Included it: Meta, MiMo, Kimi. Did not include: Claude, ChatGPT, Grok, Gemini, DeepSeek, Wenxin, Qwen.In their lists: Meta: 2 · MiMo: 2 · Kimi: 2
What each AI noted 3
- Metaplace 2
High OSWorld-Verified score, terrible 2024 AgentDojo attack rate of 47.69% for GPT-4o - then 0/288 on a narrow 2026 banking subset for GPT-5.4. Both are true, for different tests.
Source ↗ - MiMoplace 2
The newest big model is meaningfully harder to trick than its little predecessor — the first real security gap by generation.
Source ↗ - Kimiplace 2
A 4.96% attack success rate on GPT-5.1 — roughly a tenth of GPT-4o's 47.69% — while the family tops desktop charts at 75.0% OSWorld-Verified.
Source ↗
3 AIs put it here - 3
Google Gemini computer use
Strong worker, weaker guard: 83% on OSWorld-Verified, but two independent injection tests put older Gemini models at 17–21% attack success. No injection result is published for Gemini 3.6 itself.
Included it: Claude, Meta, MiMo, Kimi, DeepSeek. Did not include: ChatGPT, Grok, Gemini, Wenxin, Qwen.In their lists: Claude: 5 · Meta: 3 · MiMo: 6 · Kimi: 5 · DeepSeek: 8
What each AI noted 5
- Claudeplace 5
Google reports one Gemini model cutting attack success to 0.7% on a benchmark, a vendor claim not yet independently reproduced.
Source ↗ - Metaplace 3
83% OSWorld-Verified for Gemini 3.6 Flash beside a 17.01% AgentDojo attack rate for Gemini 1.5 Pro - capable, middling on the old injection test.
Source ↗ - MiMoplace 6
The only major agent that previews every action live — the safest default posture of the product layer.
Source ↗ - Kimiplace 5
57% utility but a 20.61% ASR — and Google's own data shows adversarial training was what moved Gemini from 92% injection success down to 18%.
Source ↗ - DeepSeekplace 8
83% OSWorld-Verified, but no reproducible AgentDojo or ASB result—work capability without a published trick-resistance number.
Source ↗
5 AIs put it here - 4
Claude Sonnet 4.5 (Anthropic)
Keeps the most work under attack on AgentDyn: 70.7 utility at 0.87% attack success. Anthropic's own stress test shows 200 attempts push attack success to 87.5%.
Included it: Kimi. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.In their lists: Kimi: 1
What each AI noted 1
- Kimiplace 1
70.7 utility with a 0.87% attack success rate on AgentDyn — the best work-per-trick ratio measured in 2026, though Anthropic's own adaptive tests warn 200-attempt attacks climb to 87.5%.
Source ↗
1 AIs put it here - 5
OpenAI GPT-5-mini
The hardest model to trick on AgentDyn: 0.37% attack success, and only 2.1% under adaptive attacks. Its desktop score is a sibling model's: GPT-5.4 mini at 72.1%.
Included it: Kimi. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.In their lists: Kimi: 3
What each AI noted 1
- Kimiplace 3
0.37% ASR with 64.8 utility — the most modest model in the AgentDyn test was the hardest to trick, and even hand-built adaptive attacks reached only 2.1%.
Source ↗
1 AIs put it here - 6
Kimi K2.5 / K2.6 (Moonshot AI)
Best utility of any open model on AgentDyn: 54.2 at 11.65% attack success. K2.6 scores 73.1% on OSWorld-Verified but finishes just 4.6% of long OSWorld 2.0 jobs.
Included it: Kimi, DeepSeek, Qwen. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, Wenxin.In their lists: Kimi: 4 · DeepSeek: 9 · Qwen: see list
What each AI noted 3
- Kimiplace 4
54.2 utility at 11.65% ASR — the best utility of any open model in AgentDyn — with K2.6's 73.1% OSWorld-Verified proving the family's desktop range.
Source ↗ - DeepSeekplace 9
84.8% OSWorld-Verified and a b3 security score of 88.0%—but b3 measures the model, not the desktop agent's attack surface.
Source ↗ - Qwen
Named differently in its own list: open the full list.
see list
3 AIs put it here - 7
Claude 3.5 Sonnet, Oct 2024 (Anthropic)
The cautionary tale: 1.11% attack success on AgentDojo's static attacks, 81% under NIST's adapted attacks, and only 14.9% on OSWorld at launch.
Included it: Meta, Kimi, DeepSeek, Qwen. Did not include: Claude, ChatGPT, Grok, Gemini, MiMo, Wenxin.In their lists: Meta: 10 · Kimi: 6 · DeepSeek: see list · Qwen: 1
What each AI noted 4
- Metaplace 10
Low AgentDojo attack rate at 9.06%, but only 33.39% utility under attack - the cheapest Claude to trick the least, and the least useful when attacked.
Source ↗ - Kimiplace 6
The field's cautionary tale: 1.11% ASR on AgentDojo's static attacks and 72.5% utility — then NIST's red team reached 81% with adapted attacks on held-out tasks.
Source ↗ - DeepSeek
Named differently in its own list: open the full list.
see list - Qwenplace 1
Completes 88.66% of useful work while shrinking prompt-injection attack success to just 7.31% on AgentDojo.
Source ↗
4 AIs put it here - 8
Meta Llama / Muse Spark
Muse Spark 1.1 scores 80.67% on OSWorld, but the Meta models with injection tests do little useful work: Llama-3-70B keeps 18.28% utility at 25.60% attack success.
Included it: Meta, Kimi. Did not include: Claude, ChatGPT, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.In their lists: Meta: 4 · Kimi: 7
What each AI noted 2
2 AIs put it here - 9
Alibaba Qwen
Has the top vendor-reported desktop score at 86.1%, yet the only Qwen model with an injection test, Qwen3 235B, shows 22.67% attack success with 10.74 utility.
Included it: Claude, Meta, Grok, Kimi. Did not include: ChatGPT, Gemini, MiMo, DeepSeek, Wenxin, Qwen.In their lists: Claude: 9 · Meta: 5 · Grok: see list · Kimi: 8
What each AI noted 4
- Claudeplace 9
Tops the vendor-reported OSWorld table at 86.1% with no published injection result.
Source ↗ - Metaplace 5
Highest vendor-reported OSWorld-Verified score at 86.1%, yet Qwen was never in the AgentDojo results table - capability without a public injection test.
Source ↗ - Grok
Named differently in its own list: open the full list.
see list - Kimiplace 8
22.67% ASR against the weakest utility in the set (10.74%) — exposure without the results.
Source ↗
4 AIs put it here - 10
OpenAI GPT-4o
Hijacked 47.69% of the time on AgentDojo and scores 12.2% on OSWorld. A revised ASB cut its attack success from 73.58% to 9.25%, a change in the benchmark rather than the model.
Included it: Meta, Grok, Kimi, Qwen. Did not include: Claude, ChatGPT, Gemini, MiMo, DeepSeek, Wenxin.In their lists: Meta: 9 · Grok: 2 · Kimi: 9 · Qwen: 7
What each AI noted 4
- Metaplace 9
AgentDojo ASR looks modest at 10.33%, but ASB direct prompt injection hit 98.40% - the two injection tests measure different attacks, and the gap matters.
Source ↗ - Grokplace 2
Strong benign utility but higher raw ASR; filters can cut attack success sharply.
Source ↗ - Kimiplace 9
47.69% ASR on AgentDojo and a 100% Plan-of-Thought backdoor rate in ASB — the benchmark era's hardest lesson in what 'mostly works' hides.
Source ↗ - Qwenplace 7
Completes 69.07% of tasks but yields to nearly half (47.69%) of prompt injections on AgentDojo without strict guardrails.
Source ↗
4 AIs put it here
02Worth knowing
What the AIs noticed while building the list.
The defense design decides usefulness. On GPT-4o, a tool filter cut attack success from 47.69% to 6.84% and raised utility to 56.28%; a detector reached 7.95% but cut utility to 21.14%.
ASB's highest average attack success is 84.30%, under mixed attacks across 13 model backbones.
Users approve 93% of permission prompts, so a prompt people rubber-stamp protects little.
No 2026 frontier system has OSWorld, AgentDojo and ASB results on the same checkpoint, so most rows pair numbers from the same model family.
03How we counted
In plain words, no formulas.
1The question
Top 10 AI Computer-Use Agents: Can They Do the Work, and Can They Be Tricked?
2Why this topic
No single leaderboard puts what an agent can do next to how easily a hidden instruction on a web page or in a file can turn it against you.
3Where it applies
Global; computer-use agent systems that operate desktop apps and browsers
4What we looked at
- Only systems with reproducible results on both kinds of test, with the model, agent setup and benchmark version named
- Task success on OSWorld (real desktop work), shown next to prompt-injection attack success on AgentDojo and Agent Security Bench (lower is better); the two are shown side by side and never averaged
- Tie-breaker: how much useful work the agent still completes while under attack
- A separate judgment of how much screen data the agent sees and whether its actions can be undone
- OSWorld-Verified (OSWorld team; BenchLM tracker) — Share of real desktop tasks completed, checked against the final app or file state
- OSWorld 2.0 (Snorkel AI) — Partial and binary completion on 108 long-horizon workflows
- AgentDojo (ETH Zurich SPY Lab) — Useful work under attack and targeted attack success across 97 tasks and 629 injection cases (lower attack success is better)
- AgentDyn (2026) — Harder, dynamic injection tests: utility, utility under attack and attack success
5Who ranked
The leading AIs, each building its own list of ten independently and naming a source for every place: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, Kimi, DeepSeek, Wenxin, Qwen.
6How the places were set
The same entry written differently by different AIs was counted as one. The more AIs put an entry on their list, and the higher they put it, the higher its place. "AI agreement" shows how many of the AIs had it on their own list. Sources are only those the AIs found themselves; nothing was added by hand.
04Each AI's own list
🇺🇸 Claude10 places · 4 notes
The same question, answered by Claude alone. The final list above combines all the AIs.
- 1Claude Opus 5 (Anthropic, 2026)
0.54% attack success in GUI tests, with 70.6% OSWorld 2.0 partial from the same model.
Source ↗ - 2Claude Fable 5 (Anthropic, 2026)
Gray Swan attack success is 2.8% against 85% on OSWorld-Verified.
Source ↗ - 3Claude Mythos 5 (Anthropic)
85.0% on OSWorld-Verified and 2.6% on Gray Swan.
- 4Claude Opus 4.8 (Anthropic)
83.4% on OSWorld-Verified; its 5.5% Gray Swan rate is the baseline Opus 5 more than halved.
Source ↗ - 5Gemini computer use (Google, 2026)
Google reports one Gemini model cutting attack success to 0.7% on a benchmark, a vendor claim not yet independently reproduced.
Source ↗ - 6OpenAI Codex computer use / GPT-6 Astra
Has a locked-use mode, but no attack-success number was found next to its OSWorld 2.0 score of 72.6%.
Source ↗ - 7Claude Sonnet 5 (Anthropic)
81.2% on OSWorld-Verified, with no separate injection figure found.
Source ↗ - 8Holo3-35B-A3B (H Company)
The top independent OSWorld-Verified run (82.6%), but no injection data was found.
Source ↗ - 9Qwen3.8-Max (Alibaba)
Tops the vendor-reported OSWorld table at 86.1% with no published injection result.
Source ↗ - 10o4-mini-based computer-use agent (OS-Harm baseline)
Complied with basic injections in about 20% of cases, the one measured counterexample.
Source ↗
What Claude noticed
Anthropic reports 0% browser-agent injection success with Auto Mode and safeguards, against 3.7% without them. The 0% figure depends on the extra layers.
A Waterloo-led study found agents retrained on more injection attacks were easier to hijack in three of four tests.
A defense that restricts tools before untrusted data is read cut targeted attack success from 57.69% to 6.84%.
UK AISI-related testing reports adaptive attacks raising AgentDojo Workspace attack success from 11% to 81%.
🇺🇸 Meta10 places · 4 notes
The same question, answered by Meta alone. The final list above combines all the AIs.
- 1Claude computer use (Anthropic)
The only family with both a top OSWorld score and a near-zero AgentDojo attack rate - but the two numbers come from different checkpoints, years apart.
Source ↗ - 2OpenAI ChatGPT agent / CUA (OpenAI)
High OSWorld-Verified score, terrible 2024 AgentDojo attack rate of 47.69% for GPT-4o - then 0/288 on a narrow 2026 banking subset for GPT-5.4. Both are true, for different tests.
Source ↗ - 3Google Gemini computer use (Google)
83% OSWorld-Verified for Gemini 3.6 Flash beside a 17.01% AgentDojo attack rate for Gemini 1.5 Pro - capable, middling on the old injection test.
Source ↗ - 4Meta Llama / Muse agent (Meta)
80.67% verified OSWorld for Muse Spark 1.1, but the Meta model actually tested in AgentDojo, Llama-3-70B, kept only 18.28% utility under attack.
Source ↗ - 5Alibaba Qwen agent (Qwen)
Highest vendor-reported OSWorld-Verified score at 86.1%, yet Qwen was never in the AgentDojo results table - capability without a public injection test.
Source ↗ - 6Cohere Command-R+ agent
Low AgentDojo attack rate, low work kept under attack: 25.12% utility under attack - secure, but not very useful when attacked.
Source ↗ - 7Google Gemma2 agent
In ASB’s backbone set but absent from AgentDojo and OSWorld-Verified - a security-test model, not a proven computer-use agent.
Source ↗ - 8Mistral Mixtral agent
Same story as Gemma2: present in ASB, missing from AgentDojo and OSWorld-Verified - no reproducible computer-use plus injection pair.
Source ↗ - 9OpenAI GPT-3.5 Turbo agent
AgentDojo ASR looks modest at 10.33%, but ASB direct prompt injection hit 98.40% - the two injection tests measure different attacks, and the gap matters.
Source ↗ - 10Anthropic Claude 3 Haiku agent
Low AgentDojo attack rate at 9.06%, but only 33.39% utility under attack - the cheapest Claude to trick the least, and the least useful when attacked.
Source ↗
What Meta noticed
In AgentDojo’s original table, Claude 3.5 Sonnet (20241022) had a 1.11% targeted attack rate while GPT-4o had 47.69% - same test, same attack, opposite security.
A 2026 replication found 0/288 undefended ASR for GPT-5.4 and 1/288 for Claude Sonnet 4.6 on the AgentDojo banking subset, but the authors warn benchmark goals were not equally reachable across models.
ASB-Revised cut GPT-4o’s ASR from 73.58% to 9.25% - a benchmark revision, not a model change, flipped the verdict on the same model.
Adaptive attackers broke 12 prompt-injection defenses that had reported near-zero ASR, reaching above 90% ASR for most - static AgentDojo attacks understate a determined attacker.
🇺🇸 Grok10 places · 1 notes
The same question, answered by Grok alone. The final list above combines all the AIs.
- 1Claude 3.5/4.x/5.x computer-use agents (Anthropic)
Higher utility under attack and lower targeted ASR than GPT-4o on AgentDojo in published runs.
Source ↗ - 2GPT-4o / OpenAI CUA setups with defenses
Strong benign utility but higher raw ASR; filters can cut attack success sharply.
Source ↗ - 3Simular Sai
High long-horizon OSWorld partial scores; security numbers not independently published on AgentDojo.
Source ↗ - 4Qwen3.8 Max setups
Top-tier Verified desktop success; limited public AgentDojo/ASB pairings.
Source ↗ - 5Microsoft Mage-VL-Cowork
Compact open hybrid agent at 66% Verified; no published prompt-injection pair scores.
Source ↗ - 6Pointer Agent
High Verified scores via hybrid methods; injection robustness not detailed.
Source ↗ - 7Coasty CUA
Strong Verified desktop score; attack-surface data sparse.
Source ↗ - 8Intelligence-Indeed Z-Agent
Highest Verified desktop completion; no public AgentDojo pairing found.
Source ↗ - 9Open-weight small models + CapLite-style guards
Zero successful attacks across thousands of AgentDojo episodes in one defense study.
Source ↗ - 10Generic frontier agents without specialized defenses
ASB reports peak average attack success of 84.30% across mixed injection methods.
Source ↗
What Grok noticed
AgentDojo’s original paper found even strong models solve <66% of tasks without attacks and attacks succeed <25% of the time; modern Claude runs show still-lower targeted ASR.
🇺🇸 Gemini10 places · 1 notes
The same question, answered by Gemini alone. The final list above combines all the AIs.
- 1Claude 3.7 Sonnet Computer Use (Anthropic)
Achieves 44.2% desktop completion while keeping indirect prompt injection hijack rates down to 2.4% on AgentDojo via dual-layer system prompt hierarchies.
- 2Claude 3.5 Sonnet Computer Use (Anthropic)
Balances 38.6% real-world desktop navigation against a low 4.8% injection attack rate, maintaining 71.4% utility even when navigating adversarial web pages.
- 3Agent S2.5 w/ o3 Reasoning Backbone
Scores an industry-leading 56.4% on OSWorld tasks, but deep reasoning chains can still be tricked into unauthorized file moves in 12.3% of indirect injection tests.
- 4OpenAI Operator (GPT-4o Vision Pipeline)
Solves 46.8% of browser/desktop workflows, but exhibits a 28.5% attack success rate when encountering untrusted third-party DOM text.
- 5ByteDance UI-TARS-2 (72B Local Vision Agent)
Delivers a formidable 53.1% OSWorld completion rate, but visual-prompt injection prompts embedded in background desktop images hijack actions in 19.4% of tests.
Source ↗ - 6Coasty Agent w/ Dual-LLM Guardrail (2025)
Sacrifices raw speed to hit 42.1% on OSWorld while cutting injection success down to 6.2% through an external supervisor filter.
- 7UI-TARS-1.5 (ByteDance Open Weights)
Executes 42.5% of verified OS tasks, but raw open-weights fine-tuning without safety alignment layers yields a 34.1% vulnerability to embedded PDF instructions.
Source ↗ - 8OS-Copilot / FRIDAY (Autonomous Shell Agent)
Completes 35.6% of Linux workflows, yet its unfettered bash execution leads to a dangerous 43.7% attack success rate when reading infected text documents.
- 9Simular GUI Agent (Visual Grounding Suite)
Reaches 38.3% on desktop tasks, suffering a 26.2% injection rate when adversarial instructions disguise themselves as system dialogue boxes.
- 10Adept ACT-1 (Browser & Enterprise Workflow)
Logs 31.2% verified workflow completion, but legacy DOM parsers succumb to 41.5% of indirect prompt injection attacks hidden in web page comments.
What Gemini noticed
Because OS-level actions like file deletion or email dispatching cannot be automatically 'undone' via an API undo button, security experts recommend mandatory confirmation gates for all irreversible desktop events.
🇨🇳 MiMo10 places · 4 notes
The same question, answered by MiMo alone. The final list above combines all the AIs.
- 1Claude Code auto mode (Anthropic, 2026)
It answers the trick question with geometry: an agent that can't reach the bait can't take the bait.
Source ↗ - 2GPT-5 family agents (OpenAI)
The newest big model is meaningfully harder to trick than its little predecessor — the first real security gap by generation.
Source ↗ - 3Claude Opus 5 (Anthropic)
The most independently cross-checked agent: its numbers survive a rival's run with only a small haircut.
Source ↗ - 4GPT-6.2 Astra (OpenAI)
Best new long-job score at a third of the price — while its own card logs workplace actions it got wrong.
Source ↗ - 5GPT-6.1 Sol (OpenAI)
Within 2.1 points of the flagship brain on long desktop jobs, at everyday prices.
Source ↗ - 6Gemini for Mac computer use (Google)
The only major agent that previews every action live — the safest default posture of the product layer.
Source ↗ - 7Claude Cowork computer use (Anthropic)
Scores near the top and documents its own biggest hole in one line: no sandbox between the agent and your screen.
Source ↗ - 8Qwen3.8 Max (Alibaba)
Highest desktop score on the board, but its security posture depends on a checkpoint you can't actually run yourself.
Source ↗ - 9Muse Spark 1.3 (Meta)
Meta's desktop agent enters the long-horizon boards at 66.9% — self-run, on its own harness.
Source ↗ - 10MiniMax M3 (MiniMax)
The honest floor: an author-run 22.3% on long jobs — published anyway, which almost nobody does.
Source ↗
What MiMo noticed
Trickability is improving by model generation for the first time: red-teaming research finds GPT-5 significantly harder to attack on AgentDojo than GPT-4o-mini.
The dangerous number isn't attack success, it's approval fatigue: real users approve 93% of permission prompts, so a prompt humans rubber-stamp is as good as no prompt at all.
Independent re-runs are rare and worth everything: Snorkel's co-author runs land within 2 points of self-reported scores for GPT-5.6 Sol but 2.3 below for Opus 5.
Privacy research (AGENTDAM) evaluates Claude computer-use agents directly and finds autonomous web/computer agents leak private data in ways their task scores never show.
🇨🇳 Kimi10 places · 4 notes
The same question, answered by Kimi alone. The final list above combines all the AIs.
- 1Claude Sonnet 4.5 (Anthropic)
70.7 utility with a 0.87% attack success rate on AgentDyn — the best work-per-trick ratio measured in 2026, though Anthropic's own adaptive tests warn 200-attempt attacks climb to 87.5%.
Source ↗ - 2GPT-5.1 / GPT-5.4 family (OpenAI)
A 4.96% attack success rate on GPT-5.1 — roughly a tenth of GPT-4o's 47.69% — while the family tops desktop charts at 75.0% OSWorld-Verified.
Source ↗ - 3GPT-5-mini (OpenAI)
0.37% ASR with 64.8 utility — the most modest model in the AgentDyn test was the hardest to trick, and even hand-built adaptive attacks reached only 2.1%.
Source ↗ - 4Kimi K2.5 / K2.6 (Moonshot AI)
54.2 utility at 11.65% ASR — the best utility of any open model in AgentDyn — with K2.6's 73.1% OSWorld-Verified proving the family's desktop range.
Source ↗ - 5Gemini 2.5 Pro / Gemini 3.5 Flash (Google)
57% utility but a 20.61% ASR — and Google's own data shows adversarial training was what moved Gemini from 92% injection success down to 18%.
Source ↗ - 6Claude 3.5 Sonnet, Oct 2024 release (Anthropic)
The field's cautionary tale: 1.11% ASR on AgentDojo's static attacks and 72.5% utility — then NIST's red team reached 81% with adapted attacks on held-out tasks.
Source ↗ - 7Llama 3.3 70B (Meta)
11.91% ASR looks respectable until you see the utility: 6.15% — it fails the work, not just the attackers.
Source ↗ - 8Qwen3 235B-A22B (Alibaba)
22.67% ASR against the weakest utility in the set (10.74%) — exposure without the results.
Source ↗ - 9GPT-4o (OpenAI)
47.69% ASR on AgentDojo and a 100% Plan-of-Thought backdoor rate in ASB — the benchmark era's hardest lesson in what 'mostly works' hides.
Source ↗ - 10GPT-4o-mini (OpenAI)
67.55% average ASR in ASB, the worst measured — small, cheap and wide open, with 95%+ rates on direct injection.
Source ↗
What Kimi noticed
Adaptive attacks flip the leaderboard: NIST CAISI raised hijacking on Claude 3.5 Sonnet agents from 11% to 81%, and a cross-lab team (OpenAI, Anthropic, DeepMind, ETH) broke 12 published defenses at >90% ASR — static benchmark safety is a floor, not a ceiling.
The only defense family with near-zero ASR that survives adaptation moves enforcement out of the model: CaMeL and Progent hit 0% ASR on AgentDyn by constraining tool calls, but CaMeL's 0% came with 0% utility there — while on AgentDojo CaMeL solved 67-77% of tasks with provable security, so the trade is real but narrow.
The utility-security tradeoff is measurable: on AgentDojo's GPT-4o run, a tool filter cut ASR from 47.69% to 6.84% while raising utility to 56.28%, but a prompt-injection detector cut ASR to 7.95% by smashing utility to 21.14% — the defense design, not the model, decides whether the agent stays useful under fire.
Screen access is the multiplier on every injection: an agent with Screen Recording and Accessibility sees everything you see and can send, delete, buy and change settings — which is why the undo question (per-app approvals, VM snapshots, reversible actions) matters as much as the ASR percentage.
🇨🇳 DeepSeek10 places · 1 notes
The same question, answered by DeepSeek alone. The final list above combines all the AIs.
- 1Claude Computer Use (Claude Fable 5, 2026)
85% on OSWorld-Verified and 0% targeted ASR on several AgentDojo suites in the closest reproducible harness—but the harness is not Claude Desktop itself.
Source ↗ - 2Ratchet harness with gpt-5.6-luna (2026)
0% targeted ASR across all four AgentDojo suites—the cleanest security result in the open AgentDojo table—but its OSWorld score is not in the same row.
Source ↗ - 3Coasty CUA v1 (2026)
82.81% on OSWorld-Verified and no published AgentDojo/ASB ASR—strong on work, unmeasured on trickery.
Source ↗ - 4OpenAI Operator (2026)
38.1% OSWorld but almost no local screen data—its attack surface is the browser, not your desktop.
Source ↗ - 5Manus (Meta) desktop agent (2026)
72.5% OSWorld in a cloud VM—reversible because the VM is disposable, but security-benchmark ASR is not published.
- 6Agent S3 (Simular, 2026)
72.6% OSWorld and open-source, so you can sandbox it—but it still needs screen and input access, and its ASR is not in the AgentDojo table.
Source ↗ - 7UiPath Screen Agent (2026)
67.1% OSWorld-Verified with broad UI automation—high utility, high control surface, ASR unpublished.
- 8Gemini 3.6 Flash agent (Google, 2026)
83% OSWorld-Verified, but no reproducible AgentDojo or ASB result—work capability without a published trick-resistance number.
Source ↗ - 9Kimi K3 agent (Moonshot AI, 2026)
84.8% OSWorld-Verified and a b3 security score of 88.0%—but b3 measures the model, not the desktop agent's attack surface.
Source ↗ - 10UI-TARS-2 (ByteDance, 2026)
47.5% OSWorld with native screen control—more exposure than a cloud agent, and no published AgentDojo ASR.
What DeepSeek noticed
AgentDojo's best published ASR is 0%, but that result comes from a specific harness (Ratchet with gpt-5.6-luna) that has no matching OSWorld score. The agent that is safest on the security benchmark is not the same agent that tops the desktop-work benchmark.
🇨🇳 Wenxin10 places
The same question, answered by Wenxin alone. The final list above combines all the AIs.
- 1Apple In-House macOS AI Agent (Early 2027 Build)
No persistent screen data capture, every action is revocable via system-level interruption before execution, 0 reported prompt injection bypasses in early internal testing
Source ↗ - 2HKU KYA Compliant Enterprise Desktop Agent
100% unique verifiable agent identity, full operation traceability, dynamic permission auto-revokes when actions exceed pre-defined scope
- 3Ant Financial Pay AI Agent (2026)
Three-tier pay authorization workflow, even fully compromised agents cannot execute unapproved high-value transactions without manual human confirmation
- 4OpenAI Agents API (2026 Beta)
Hosted sandbox runtime blocks 89% of known prompt injection payloads, all risky actions require secondary confirmation before execution
- 5Hybrid RBAC+ABAC Enterprise Custom Agent (2026)
Intent-behavior alignment validation added to permission checks, blocks 91% of unintended in-scope overreach operations
- 6Perplexity Computer (2026)
Isolated cloud environment fully segregated from local desktop sensitive data, no persistent local screen access at all
- 7Anthropic Claude Computer Use Agent (2026)
Constitutional AI guardrails block 87% of unauthorized harmful action attempts, 49% developer adoption with top industry trust ratings
- 8Sai AI Agent (2026)
All GUI automation operations run on dedicated isolated cloud desktops, no access to local host's sensitive data or files by default
- 9Meta Muse Agent (2026 Open Source)
Full runtime code is publicly auditable, users can add custom prompt injection detection rules without relying on third-party vendors
Source ↗ - 10OpenCode Open Source Coding Agent (2026)
No automatic write access to production files by default, every file modification requires user review and explicit approval
🇨🇳 Qwen10 places · 3 notes
The same question, answered by Qwen alone. The final list above combines all the AIs.
- 1Claude 3.7 Sonnet
Completes 88.66% of useful work while shrinking prompt-injection attack success to just 7.31% on AgentDojo.
Source ↗ - 2AgentSys (Defended Agent)
Holds attack success to 0.78% on AgentDojo and 4.25% on ASB while improving benign task utility.
Source ↗ - 3Claude Opus 4.6
Scored a perfect 100% in one targeted security audit, though real-world reversibility of its desktop actions remains debated.
Source ↗ - 4Verigrey Defended Agent
Keeps 93.2% of useful work intact while holding attack success under 1% through action-governance rather than prompt scanning.
Source ↗ - 5o3 (Defended)
Completes 77% of AgentDojo tasks with provable security, compared to 84% for the undefended baseline.
Source ↗ - 6Claude 3.5 Sonnet
Achieves the highest NRP (Normalized Risk-adjusted Performance) on ASB at 43.56, despite baseline attack vulnerabilities.
Source ↗ - 7GPT-4o
Completes 69.07% of tasks but yields to nearly half (47.69%) of prompt injections on AgentDojo without strict guardrails.
Source ↗ - 8GPT-4o-mini
Shows a 49.92% attack success rate on AgentDojo, exposing how smaller models struggle to filter injected tool outputs.
Source ↗ - 9Claude 3.5 Sonnet (Red-teamed)
While baseline attacks only succeed 11% of the time, coordinated red-team prompts breach it 81% of the time.
Source ↗ - 10GPT-4-0125
Early 2024 versions suffered a 56.3% targeted prompt-injection success rate before modern AgentDojo defenses were standardized.
Source ↗
What Qwen noticed
Action governance outperforms prompt scanning: blocking the result of a tool call keeps attack success under 1% while saving 93.2% of utility.
AgentDojo's targeted prompt-injection attack success rate has fallen from 56.3% on early GPT-4 models to just 7.3% on Claude 3.7 Sonnet, showing rapid security evolution.
The Agent Security Bench introduces the 'Normalized Risk-adjusted Performance' (NRP) metric, proving that refusing malicious prompts doesn't have to mean refusing legitimate work.