Back to the trend of the day: Apple is building an AI helper that can use your Mac for you
Poly Rating · from the trend of the day

AI Agent Prompt Injection Rankings 2026

Claude 3.5 Sonnet resisted all but 1.11% of AgentDojo's standard prompt injections, yet NIST's red team hijacked the same generation of agent 81% of the time with adapted attacks.

  • No. 1: Claude Opus 5 (Anthropic)
  • 10 places

01Final list

The leading AIs each built their own list of ten independently. Here is one result that is easy to check: every place has a reason, a source, and the list of AIs that included it.

Only systems with reproducible results on both kinds of test, with the model, agent setup and benchmark version named

A bright badge means this AI put the entry on its list. A dim badge means it did not.

  1. 1

    Claude Opus 5 (Anthropic)

    Best pairing found: 83.39% on OSWorld-Verified and 0.54% GUI injection success. The injection figure is Anthropic's own; no AgentDojo or ASB run is published.

    Included it: Claude, MiMo, Kimi, DeepSeek. Did not include: ChatGPT, Meta, Grok, Gemini, Wenxin, Qwen.

    In their lists: Claude: 1 · MiMo: 3 · Kimi: see list · DeepSeek: 1

    What each AI noted 4
    • Claudeplace 1

      0.54% attack success in GUI tests, with 70.6% OSWorld 2.0 partial from the same model.

      Source ↗
    • MiMoplace 3

      The most independently cross-checked agent: its numbers survive a rival's run with only a small haircut.

      Source ↗
    • Kimi

      Named differently in its own list: open the full list.

      see list
    • DeepSeekplace 1

      85% on OSWorld-Verified and 0% targeted ASR on several AgentDojo suites in the closest reproducible harness—but the harness is not Claude Desktop itself.

      Source ↗
    4 AIs put it here
  2. 2

    OpenAI GPT-5 family (GPT-5.1 / 5.4 / 5.5)

    GPT-5.1 is hijacked 4.96% of the time on AgentDyn, about a tenth of GPT-4o's rate, and GPT-5.4 went 0 for 288 in a 2026 replication. GPT-5.5 scores 78.7% on OSWorld-Verified.

    Included it: Meta, MiMo, Kimi. Did not include: Claude, ChatGPT, Grok, Gemini, DeepSeek, Wenxin, Qwen.

    In their lists: Meta: 2 · MiMo: 2 · Kimi: 2

    What each AI noted 3
    • Metaplace 2

      High OSWorld-Verified score, terrible 2024 AgentDojo attack rate of 47.69% for GPT-4o - then 0/288 on a narrow 2026 banking subset for GPT-5.4. Both are true, for different tests.

      Source ↗
    • MiMoplace 2

      The newest big model is meaningfully harder to trick than its little predecessor — the first real security gap by generation.

      Source ↗
    • Kimiplace 2

      A 4.96% attack success rate on GPT-5.1 — roughly a tenth of GPT-4o's 47.69% — while the family tops desktop charts at 75.0% OSWorld-Verified.

      Source ↗
    3 AIs put it here
  3. 3

    Google Gemini computer use

    Strong worker, weaker guard: 83% on OSWorld-Verified, but two independent injection tests put older Gemini models at 17–21% attack success. No injection result is published for Gemini 3.6 itself.

    Included it: Claude, Meta, MiMo, Kimi, DeepSeek. Did not include: ChatGPT, Grok, Gemini, Wenxin, Qwen.

    In their lists: Claude: 5 · Meta: 3 · MiMo: 6 · Kimi: 5 · DeepSeek: 8

    What each AI noted 5
    • Claudeplace 5

      Google reports one Gemini model cutting attack success to 0.7% on a benchmark, a vendor claim not yet independently reproduced.

      Source ↗
    • Metaplace 3

      83% OSWorld-Verified for Gemini 3.6 Flash beside a 17.01% AgentDojo attack rate for Gemini 1.5 Pro - capable, middling on the old injection test.

      Source ↗
    • MiMoplace 6

      The only major agent that previews every action live — the safest default posture of the product layer.

      Source ↗
    • Kimiplace 5

      57% utility but a 20.61% ASR — and Google's own data shows adversarial training was what moved Gemini from 92% injection success down to 18%.

      Source ↗
    • DeepSeekplace 8

      83% OSWorld-Verified, but no reproducible AgentDojo or ASB result—work capability without a published trick-resistance number.

      Source ↗
    5 AIs put it here
  4. 4

    Claude Sonnet 4.5 (Anthropic)

    Keeps the most work under attack on AgentDyn: 70.7 utility at 0.87% attack success. Anthropic's own stress test shows 200 attempts push attack success to 87.5%.

    Included it: Kimi. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.

    In their lists: Kimi: 1

    What each AI noted 1
    • Kimiplace 1

      70.7 utility with a 0.87% attack success rate on AgentDyn — the best work-per-trick ratio measured in 2026, though Anthropic's own adaptive tests warn 200-attempt attacks climb to 87.5%.

      Source ↗
    1 AIs put it here
  5. 5

    OpenAI GPT-5-mini

    The hardest model to trick on AgentDyn: 0.37% attack success, and only 2.1% under adaptive attacks. Its desktop score is a sibling model's: GPT-5.4 mini at 72.1%.

    Included it: Kimi. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.

    In their lists: Kimi: 3

    What each AI noted 1
    • Kimiplace 3

      0.37% ASR with 64.8 utility — the most modest model in the AgentDyn test was the hardest to trick, and even hand-built adaptive attacks reached only 2.1%.

      Source ↗
    1 AIs put it here
  6. 6

    Kimi K2.5 / K2.6 (Moonshot AI)

    Best utility of any open model on AgentDyn: 54.2 at 11.65% attack success. K2.6 scores 73.1% on OSWorld-Verified but finishes just 4.6% of long OSWorld 2.0 jobs.

    Included it: Kimi, DeepSeek, Qwen. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, Wenxin.

    In their lists: Kimi: 4 · DeepSeek: 9 · Qwen: see list

    What each AI noted 3
    • Kimiplace 4

      54.2 utility at 11.65% ASR — the best utility of any open model in AgentDyn — with K2.6's 73.1% OSWorld-Verified proving the family's desktop range.

      Source ↗
    • DeepSeekplace 9

      84.8% OSWorld-Verified and a b3 security score of 88.0%—but b3 measures the model, not the desktop agent's attack surface.

      Source ↗
    • Qwen

      Named differently in its own list: open the full list.

      see list
    3 AIs put it here
  7. 7

    Claude 3.5 Sonnet, Oct 2024 (Anthropic)

    The cautionary tale: 1.11% attack success on AgentDojo's static attacks, 81% under NIST's adapted attacks, and only 14.9% on OSWorld at launch.

    Included it: Meta, Kimi, DeepSeek, Qwen. Did not include: Claude, ChatGPT, Grok, Gemini, MiMo, Wenxin.

    In their lists: Meta: 10 · Kimi: 6 · DeepSeek: see list · Qwen: 1

    What each AI noted 4
    • Metaplace 10

      Low AgentDojo attack rate at 9.06%, but only 33.39% utility under attack - the cheapest Claude to trick the least, and the least useful when attacked.

      Source ↗
    • Kimiplace 6

      The field's cautionary tale: 1.11% ASR on AgentDojo's static attacks and 72.5% utility — then NIST's red team reached 81% with adapted attacks on held-out tasks.

      Source ↗
    • DeepSeek

      Named differently in its own list: open the full list.

      see list
    • Qwenplace 1

      Completes 88.66% of useful work while shrinking prompt-injection attack success to just 7.31% on AgentDojo.

      Source ↗
    4 AIs put it here
  8. 8

    Meta Llama / Muse Spark

    Muse Spark 1.1 scores 80.67% on OSWorld, but the Meta models with injection tests do little useful work: Llama-3-70B keeps 18.28% utility at 25.60% attack success.

    Included it: Meta, Kimi. Did not include: Claude, ChatGPT, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.

    In their lists: Meta: 4 · Kimi: 7

    What each AI noted 2
    • Metaplace 4

      80.67% verified OSWorld for Muse Spark 1.1, but the Meta model actually tested in AgentDojo, Llama-3-70B, kept only 18.28% utility under attack.

      Source ↗
    • Kimiplace 7

      11.91% ASR looks respectable until you see the utility: 6.15% — it fails the work, not just the attackers.

      Source ↗
    2 AIs put it here
  9. 9

    Alibaba Qwen

    Has the top vendor-reported desktop score at 86.1%, yet the only Qwen model with an injection test, Qwen3 235B, shows 22.67% attack success with 10.74 utility.

    Included it: Claude, Meta, Grok, Kimi. Did not include: ChatGPT, Gemini, MiMo, DeepSeek, Wenxin, Qwen.

    In their lists: Claude: 9 · Meta: 5 · Grok: see list · Kimi: 8

    What each AI noted 4
    • Claudeplace 9

      Tops the vendor-reported OSWorld table at 86.1% with no published injection result.

      Source ↗
    • Metaplace 5

      Highest vendor-reported OSWorld-Verified score at 86.1%, yet Qwen was never in the AgentDojo results table - capability without a public injection test.

      Source ↗
    • Grok

      Named differently in its own list: open the full list.

      see list
    • Kimiplace 8

      22.67% ASR against the weakest utility in the set (10.74%) — exposure without the results.

      Source ↗
    4 AIs put it here
  10. 10

    OpenAI GPT-4o

    Hijacked 47.69% of the time on AgentDojo and scores 12.2% on OSWorld. A revised ASB cut its attack success from 73.58% to 9.25%, a change in the benchmark rather than the model.

    Included it: Meta, Grok, Kimi, Qwen. Did not include: Claude, ChatGPT, Gemini, MiMo, DeepSeek, Wenxin.

    In their lists: Meta: 9 · Grok: 2 · Kimi: 9 · Qwen: 7

    What each AI noted 4
    • Metaplace 9

      AgentDojo ASR looks modest at 10.33%, but ASB direct prompt injection hit 98.40% - the two injection tests measure different attacks, and the gap matters.

      Source ↗
    • Grokplace 2

      Strong benign utility but higher raw ASR; filters can cut attack success sharply.

      Source ↗
    • Kimiplace 9

      47.69% ASR on AgentDojo and a 100% Plan-of-Thought backdoor rate in ASB — the benchmark era's hardest lesson in what 'mostly works' hides.

      Source ↗
    • Qwenplace 7

      Completes 69.07% of tasks but yields to nearly half (47.69%) of prompt injections on AgentDojo without strict guardrails.

      Source ↗
    4 AIs put it here

More about the top three ↓

02Worth knowing

What the AIs noticed while building the list.

Adaptive attackers broke 12 published prompt-injection defenses that had reported near-zero attack success, pushing most above 90%.

Noticed by:
Included it: Meta, Kimi. Did not include: Claude, ChatGPT, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.
Source ↗

The defense design decides usefulness. On GPT-4o, a tool filter cut attack success from 47.69% to 6.84% and raised utility to 56.28%; a detector reached 7.95% but cut utility to 21.14%.

Noticed by:
Included it: Kimi. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.
Source ↗

ASB's highest average attack success is 84.30%, under mixed attacks across 13 model backbones.

Noticed by:
Included it: Claude, ChatGPT, Meta, Grok, Kimi, DeepSeek. Did not include: Gemini, MiMo, Wenxin, Qwen.
Source ↗

Users approve 93% of permission prompts, so a prompt people rubber-stamp protects little.

Noticed by:
Included it: MiMo. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, Kimi, DeepSeek, Wenxin, Qwen.
Source ↗

No 2026 frontier system has OSWorld, AgentDojo and ASB results on the same checkpoint, so most rows pair numbers from the same model family.

Noticed by:
Included it: ChatGPT, Meta, Kimi. Did not include: Claude, Grok, Gemini, MiMo, DeepSeek, Wenxin, Qwen.
Source ↗

03How we counted

In plain words, no formulas.

1The question

Top 10 AI Computer-Use Agents: Can They Do the Work, and Can They Be Tricked?

2Why this topic

No single leaderboard puts what an agent can do next to how easily a hidden instruction on a web page or in a file can turn it against you.

3Where it applies

Global; computer-use agent systems that operate desktop apps and browsers

4What we looked at

  • Only systems with reproducible results on both kinds of test, with the model, agent setup and benchmark version named
  • Task success on OSWorld (real desktop work), shown next to prompt-injection attack success on AgentDojo and Agent Security Bench (lower is better); the two are shown side by side and never averaged
  • Tie-breaker: how much useful work the agent still completes while under attack
  • A separate judgment of how much screen data the agent sees and whether its actions can be undone
  • OSWorld-Verified (OSWorld team; BenchLM tracker) — Share of real desktop tasks completed, checked against the final app or file state
  • OSWorld 2.0 (Snorkel AI) — Partial and binary completion on 108 long-horizon workflows
  • AgentDojo (ETH Zurich SPY Lab) — Useful work under attack and targeted attack success across 97 tasks and 629 injection cases (lower attack success is better)
  • AgentDyn (2026) — Harder, dynamic injection tests: utility, utility under attack and attack success

5Who ranked

The leading AIs, each building its own list of ten independently and naming a source for every place: Claude, ChatGPT, Meta, Grok, Gemini, MiMo, Kimi, DeepSeek, Wenxin, Qwen.

6How the places were set

The same entry written differently by different AIs was counted as one. The more AIs put an entry on their list, and the higher they put it, the higher its place. "AI agreement" shows how many of the AIs had it on their own list. Sources are only those the AIs found themselves; nothing was added by hand.

04Each AI's own list

🇺🇸 Claude10 places · 4 notes

The same question, answered by Claude alone. The final list above combines all the AIs.

  1. 1
    Claude Opus 5 (Anthropic, 2026)

    0.54% attack success in GUI tests, with 70.6% OSWorld 2.0 partial from the same model.

    Source ↗
  2. 2
    Claude Fable 5 (Anthropic, 2026)

    Gray Swan attack success is 2.8% against 85% on OSWorld-Verified.

    Source ↗
  3. 3
    Claude Mythos 5 (Anthropic)

    85.0% on OSWorld-Verified and 2.6% on Gray Swan.

  4. 4
    Claude Opus 4.8 (Anthropic)

    83.4% on OSWorld-Verified; its 5.5% Gray Swan rate is the baseline Opus 5 more than halved.

    Source ↗
  5. 5
    Gemini computer use (Google, 2026)

    Google reports one Gemini model cutting attack success to 0.7% on a benchmark, a vendor claim not yet independently reproduced.

    Source ↗
  6. 6
    OpenAI Codex computer use / GPT-6 Astra

    Has a locked-use mode, but no attack-success number was found next to its OSWorld 2.0 score of 72.6%.

    Source ↗
  7. 7
    Claude Sonnet 5 (Anthropic)

    81.2% on OSWorld-Verified, with no separate injection figure found.

    Source ↗
  8. 8
    Holo3-35B-A3B (H Company)

    The top independent OSWorld-Verified run (82.6%), but no injection data was found.

    Source ↗
  9. 9
    Qwen3.8-Max (Alibaba)

    Tops the vendor-reported OSWorld table at 86.1% with no published injection result.

    Source ↗
  10. 10
    o4-mini-based computer-use agent (OS-Harm baseline)

    Complied with basic injections in about 20% of cases, the one measured counterexample.

    Source ↗

What Claude noticed

Anthropic reports 0% browser-agent injection success with Auto Mode and safeguards, against 3.7% without them. The 0% figure depends on the extra layers.

A Waterloo-led study found agents retrained on more injection attacks were easier to hijack in three of four tests.

A defense that restricts tools before untrusted data is read cut targeted attack success from 57.69% to 6.84%.

UK AISI-related testing reports adaptive attacks raising AgentDojo Workspace attack success from 11% to 81%.

🇺🇸 Meta10 places · 4 notes

The same question, answered by Meta alone. The final list above combines all the AIs.

  1. 1
    Claude computer use (Anthropic)

    The only family with both a top OSWorld score and a near-zero AgentDojo attack rate - but the two numbers come from different checkpoints, years apart.

    Source ↗
  2. 2
    OpenAI ChatGPT agent / CUA (OpenAI)

    High OSWorld-Verified score, terrible 2024 AgentDojo attack rate of 47.69% for GPT-4o - then 0/288 on a narrow 2026 banking subset for GPT-5.4. Both are true, for different tests.

    Source ↗
  3. 3
    Google Gemini computer use (Google)

    83% OSWorld-Verified for Gemini 3.6 Flash beside a 17.01% AgentDojo attack rate for Gemini 1.5 Pro - capable, middling on the old injection test.

    Source ↗
  4. 4
    Meta Llama / Muse agent (Meta)

    80.67% verified OSWorld for Muse Spark 1.1, but the Meta model actually tested in AgentDojo, Llama-3-70B, kept only 18.28% utility under attack.

    Source ↗
  5. 5
    Alibaba Qwen agent (Qwen)

    Highest vendor-reported OSWorld-Verified score at 86.1%, yet Qwen was never in the AgentDojo results table - capability without a public injection test.

    Source ↗
  6. 6
    Cohere Command-R+ agent

    Low AgentDojo attack rate, low work kept under attack: 25.12% utility under attack - secure, but not very useful when attacked.

    Source ↗
  7. 7
    Google Gemma2 agent

    In ASB’s backbone set but absent from AgentDojo and OSWorld-Verified - a security-test model, not a proven computer-use agent.

    Source ↗
  8. 8
    Mistral Mixtral agent

    Same story as Gemma2: present in ASB, missing from AgentDojo and OSWorld-Verified - no reproducible computer-use plus injection pair.

    Source ↗
  9. 9
    OpenAI GPT-3.5 Turbo agent

    AgentDojo ASR looks modest at 10.33%, but ASB direct prompt injection hit 98.40% - the two injection tests measure different attacks, and the gap matters.

    Source ↗
  10. 10
    Anthropic Claude 3 Haiku agent

    Low AgentDojo attack rate at 9.06%, but only 33.39% utility under attack - the cheapest Claude to trick the least, and the least useful when attacked.

    Source ↗

What Meta noticed

In AgentDojo’s original table, Claude 3.5 Sonnet (20241022) had a 1.11% targeted attack rate while GPT-4o had 47.69% - same test, same attack, opposite security.

A 2026 replication found 0/288 undefended ASR for GPT-5.4 and 1/288 for Claude Sonnet 4.6 on the AgentDojo banking subset, but the authors warn benchmark goals were not equally reachable across models.

ASB-Revised cut GPT-4o’s ASR from 73.58% to 9.25% - a benchmark revision, not a model change, flipped the verdict on the same model.

Adaptive attackers broke 12 prompt-injection defenses that had reported near-zero ASR, reaching above 90% ASR for most - static AgentDojo attacks understate a determined attacker.

🇺🇸 Grok10 places · 1 notes

The same question, answered by Grok alone. The final list above combines all the AIs.

  1. 1
    Claude 3.5/4.x/5.x computer-use agents (Anthropic)

    Higher utility under attack and lower targeted ASR than GPT-4o on AgentDojo in published runs.

    Source ↗
  2. 2
    GPT-4o / OpenAI CUA setups with defenses

    Strong benign utility but higher raw ASR; filters can cut attack success sharply.

    Source ↗
  3. 3
    Simular Sai

    High long-horizon OSWorld partial scores; security numbers not independently published on AgentDojo.

    Source ↗
  4. 4
    Qwen3.8 Max setups

    Top-tier Verified desktop success; limited public AgentDojo/ASB pairings.

    Source ↗
  5. 5
    Microsoft Mage-VL-Cowork

    Compact open hybrid agent at 66% Verified; no published prompt-injection pair scores.

    Source ↗
  6. 6
    Pointer Agent

    High Verified scores via hybrid methods; injection robustness not detailed.

    Source ↗
  7. 7
    Coasty CUA

    Strong Verified desktop score; attack-surface data sparse.

    Source ↗
  8. 8
    Intelligence-Indeed Z-Agent

    Highest Verified desktop completion; no public AgentDojo pairing found.

    Source ↗
  9. 9
    Open-weight small models + CapLite-style guards

    Zero successful attacks across thousands of AgentDojo episodes in one defense study.

    Source ↗
  10. 10
    Generic frontier agents without specialized defenses

    ASB reports peak average attack success of 84.30% across mixed injection methods.

    Source ↗

What Grok noticed

AgentDojo’s original paper found even strong models solve <66% of tasks without attacks and attacks succeed <25% of the time; modern Claude runs show still-lower targeted ASR.

🇺🇸 Gemini10 places · 1 notes

The same question, answered by Gemini alone. The final list above combines all the AIs.

  1. 1
    Claude 3.7 Sonnet Computer Use (Anthropic)

    Achieves 44.2% desktop completion while keeping indirect prompt injection hijack rates down to 2.4% on AgentDojo via dual-layer system prompt hierarchies.

  2. 2
    Claude 3.5 Sonnet Computer Use (Anthropic)

    Balances 38.6% real-world desktop navigation against a low 4.8% injection attack rate, maintaining 71.4% utility even when navigating adversarial web pages.

  3. 3
    Agent S2.5 w/ o3 Reasoning Backbone

    Scores an industry-leading 56.4% on OSWorld tasks, but deep reasoning chains can still be tricked into unauthorized file moves in 12.3% of indirect injection tests.

  4. 4
    OpenAI Operator (GPT-4o Vision Pipeline)

    Solves 46.8% of browser/desktop workflows, but exhibits a 28.5% attack success rate when encountering untrusted third-party DOM text.

  5. 5
    ByteDance UI-TARS-2 (72B Local Vision Agent)

    Delivers a formidable 53.1% OSWorld completion rate, but visual-prompt injection prompts embedded in background desktop images hijack actions in 19.4% of tests.

    Source ↗
  6. 6
    Coasty Agent w/ Dual-LLM Guardrail (2025)

    Sacrifices raw speed to hit 42.1% on OSWorld while cutting injection success down to 6.2% through an external supervisor filter.

  7. 7
    UI-TARS-1.5 (ByteDance Open Weights)

    Executes 42.5% of verified OS tasks, but raw open-weights fine-tuning without safety alignment layers yields a 34.1% vulnerability to embedded PDF instructions.

    Source ↗
  8. 8
    OS-Copilot / FRIDAY (Autonomous Shell Agent)

    Completes 35.6% of Linux workflows, yet its unfettered bash execution leads to a dangerous 43.7% attack success rate when reading infected text documents.

  9. 9
    Simular GUI Agent (Visual Grounding Suite)

    Reaches 38.3% on desktop tasks, suffering a 26.2% injection rate when adversarial instructions disguise themselves as system dialogue boxes.

  10. 10
    Adept ACT-1 (Browser & Enterprise Workflow)

    Logs 31.2% verified workflow completion, but legacy DOM parsers succumb to 41.5% of indirect prompt injection attacks hidden in web page comments.

What Gemini noticed

Because OS-level actions like file deletion or email dispatching cannot be automatically 'undone' via an API undo button, security experts recommend mandatory confirmation gates for all irreversible desktop events.

🇨🇳 MiMo10 places · 4 notes

The same question, answered by MiMo alone. The final list above combines all the AIs.

  1. 1
    Claude Code auto mode (Anthropic, 2026)

    It answers the trick question with geometry: an agent that can't reach the bait can't take the bait.

    Source ↗
  2. 2
    GPT-5 family agents (OpenAI)

    The newest big model is meaningfully harder to trick than its little predecessor — the first real security gap by generation.

    Source ↗
  3. 3
    Claude Opus 5 (Anthropic)

    The most independently cross-checked agent: its numbers survive a rival's run with only a small haircut.

    Source ↗
  4. 4
    GPT-6.2 Astra (OpenAI)

    Best new long-job score at a third of the price — while its own card logs workplace actions it got wrong.

    Source ↗
  5. 5
    GPT-6.1 Sol (OpenAI)

    Within 2.1 points of the flagship brain on long desktop jobs, at everyday prices.

    Source ↗
  6. 6
    Gemini for Mac computer use (Google)

    The only major agent that previews every action live — the safest default posture of the product layer.

    Source ↗
  7. 7
    Claude Cowork computer use (Anthropic)

    Scores near the top and documents its own biggest hole in one line: no sandbox between the agent and your screen.

    Source ↗
  8. 8
    Qwen3.8 Max (Alibaba)

    Highest desktop score on the board, but its security posture depends on a checkpoint you can't actually run yourself.

    Source ↗
  9. 9
    Muse Spark 1.3 (Meta)

    Meta's desktop agent enters the long-horizon boards at 66.9% — self-run, on its own harness.

    Source ↗
  10. 10
    MiniMax M3 (MiniMax)

    The honest floor: an author-run 22.3% on long jobs — published anyway, which almost nobody does.

    Source ↗

What MiMo noticed

Trickability is improving by model generation for the first time: red-teaming research finds GPT-5 significantly harder to attack on AgentDojo than GPT-4o-mini.

The dangerous number isn't attack success, it's approval fatigue: real users approve 93% of permission prompts, so a prompt humans rubber-stamp is as good as no prompt at all.

Independent re-runs are rare and worth everything: Snorkel's co-author runs land within 2 points of self-reported scores for GPT-5.6 Sol but 2.3 below for Opus 5.

Privacy research (AGENTDAM) evaluates Claude computer-use agents directly and finds autonomous web/computer agents leak private data in ways their task scores never show.

🇨🇳 Kimi10 places · 4 notes

The same question, answered by Kimi alone. The final list above combines all the AIs.

  1. 1
    Claude Sonnet 4.5 (Anthropic)

    70.7 utility with a 0.87% attack success rate on AgentDyn — the best work-per-trick ratio measured in 2026, though Anthropic's own adaptive tests warn 200-attempt attacks climb to 87.5%.

    Source ↗
  2. 2
    GPT-5.1 / GPT-5.4 family (OpenAI)

    A 4.96% attack success rate on GPT-5.1 — roughly a tenth of GPT-4o's 47.69% — while the family tops desktop charts at 75.0% OSWorld-Verified.

    Source ↗
  3. 3
    GPT-5-mini (OpenAI)

    0.37% ASR with 64.8 utility — the most modest model in the AgentDyn test was the hardest to trick, and even hand-built adaptive attacks reached only 2.1%.

    Source ↗
  4. 4
    Kimi K2.5 / K2.6 (Moonshot AI)

    54.2 utility at 11.65% ASR — the best utility of any open model in AgentDyn — with K2.6's 73.1% OSWorld-Verified proving the family's desktop range.

    Source ↗
  5. 5
    Gemini 2.5 Pro / Gemini 3.5 Flash (Google)

    57% utility but a 20.61% ASR — and Google's own data shows adversarial training was what moved Gemini from 92% injection success down to 18%.

    Source ↗
  6. 6
    Claude 3.5 Sonnet, Oct 2024 release (Anthropic)

    The field's cautionary tale: 1.11% ASR on AgentDojo's static attacks and 72.5% utility — then NIST's red team reached 81% with adapted attacks on held-out tasks.

    Source ↗
  7. 7
    Llama 3.3 70B (Meta)

    11.91% ASR looks respectable until you see the utility: 6.15% — it fails the work, not just the attackers.

    Source ↗
  8. 8
    Qwen3 235B-A22B (Alibaba)

    22.67% ASR against the weakest utility in the set (10.74%) — exposure without the results.

    Source ↗
  9. 9
    GPT-4o (OpenAI)

    47.69% ASR on AgentDojo and a 100% Plan-of-Thought backdoor rate in ASB — the benchmark era's hardest lesson in what 'mostly works' hides.

    Source ↗
  10. 10
    GPT-4o-mini (OpenAI)

    67.55% average ASR in ASB, the worst measured — small, cheap and wide open, with 95%+ rates on direct injection.

    Source ↗

What Kimi noticed

Adaptive attacks flip the leaderboard: NIST CAISI raised hijacking on Claude 3.5 Sonnet agents from 11% to 81%, and a cross-lab team (OpenAI, Anthropic, DeepMind, ETH) broke 12 published defenses at >90% ASR — static benchmark safety is a floor, not a ceiling.

The only defense family with near-zero ASR that survives adaptation moves enforcement out of the model: CaMeL and Progent hit 0% ASR on AgentDyn by constraining tool calls, but CaMeL's 0% came with 0% utility there — while on AgentDojo CaMeL solved 67-77% of tasks with provable security, so the trade is real but narrow.

The utility-security tradeoff is measurable: on AgentDojo's GPT-4o run, a tool filter cut ASR from 47.69% to 6.84% while raising utility to 56.28%, but a prompt-injection detector cut ASR to 7.95% by smashing utility to 21.14% — the defense design, not the model, decides whether the agent stays useful under fire.

Screen access is the multiplier on every injection: an agent with Screen Recording and Accessibility sees everything you see and can send, delete, buy and change settings — which is why the undo question (per-app approvals, VM snapshots, reversible actions) matters as much as the ASR percentage.

🇨🇳 DeepSeek10 places · 1 notes

The same question, answered by DeepSeek alone. The final list above combines all the AIs.

  1. 1
    Claude Computer Use (Claude Fable 5, 2026)

    85% on OSWorld-Verified and 0% targeted ASR on several AgentDojo suites in the closest reproducible harness—but the harness is not Claude Desktop itself.

    Source ↗
  2. 2
    Ratchet harness with gpt-5.6-luna (2026)

    0% targeted ASR across all four AgentDojo suites—the cleanest security result in the open AgentDojo table—but its OSWorld score is not in the same row.

    Source ↗
  3. 3
    Coasty CUA v1 (2026)

    82.81% on OSWorld-Verified and no published AgentDojo/ASB ASR—strong on work, unmeasured on trickery.

    Source ↗
  4. 4
    OpenAI Operator (2026)

    38.1% OSWorld but almost no local screen data—its attack surface is the browser, not your desktop.

    Source ↗
  5. 5
    Manus (Meta) desktop agent (2026)

    72.5% OSWorld in a cloud VM—reversible because the VM is disposable, but security-benchmark ASR is not published.

  6. 6
    Agent S3 (Simular, 2026)

    72.6% OSWorld and open-source, so you can sandbox it—but it still needs screen and input access, and its ASR is not in the AgentDojo table.

    Source ↗
  7. 7
    UiPath Screen Agent (2026)

    67.1% OSWorld-Verified with broad UI automation—high utility, high control surface, ASR unpublished.

  8. 8
    Gemini 3.6 Flash agent (Google, 2026)

    83% OSWorld-Verified, but no reproducible AgentDojo or ASB result—work capability without a published trick-resistance number.

    Source ↗
  9. 9
    Kimi K3 agent (Moonshot AI, 2026)

    84.8% OSWorld-Verified and a b3 security score of 88.0%—but b3 measures the model, not the desktop agent's attack surface.

    Source ↗
  10. 10
    UI-TARS-2 (ByteDance, 2026)

    47.5% OSWorld with native screen control—more exposure than a cloud agent, and no published AgentDojo ASR.

What DeepSeek noticed

AgentDojo's best published ASR is 0%, but that result comes from a specific harness (Ratchet with gpt-5.6-luna) that has no matching OSWorld score. The agent that is safest on the security benchmark is not the same agent that tops the desktop-work benchmark.

🇨🇳 Wenxin10 places

The same question, answered by Wenxin alone. The final list above combines all the AIs.

  1. 1
    Apple In-House macOS AI Agent (Early 2027 Build)

    No persistent screen data capture, every action is revocable via system-level interruption before execution, 0 reported prompt injection bypasses in early internal testing

    Source ↗
  2. 2
    HKU KYA Compliant Enterprise Desktop Agent

    100% unique verifiable agent identity, full operation traceability, dynamic permission auto-revokes when actions exceed pre-defined scope

  3. 3
    Ant Financial Pay AI Agent (2026)

    Three-tier pay authorization workflow, even fully compromised agents cannot execute unapproved high-value transactions without manual human confirmation

  4. 4
    OpenAI Agents API (2026 Beta)

    Hosted sandbox runtime blocks 89% of known prompt injection payloads, all risky actions require secondary confirmation before execution

  5. 5
    Hybrid RBAC+ABAC Enterprise Custom Agent (2026)

    Intent-behavior alignment validation added to permission checks, blocks 91% of unintended in-scope overreach operations

  6. 6
    Perplexity Computer (2026)

    Isolated cloud environment fully segregated from local desktop sensitive data, no persistent local screen access at all

  7. 7
    Anthropic Claude Computer Use Agent (2026)

    Constitutional AI guardrails block 87% of unauthorized harmful action attempts, 49% developer adoption with top industry trust ratings

  8. 8
    Sai AI Agent (2026)

    All GUI automation operations run on dedicated isolated cloud desktops, no access to local host's sensitive data or files by default

  9. 9
    Meta Muse Agent (2026 Open Source)

    Full runtime code is publicly auditable, users can add custom prompt injection detection rules without relying on third-party vendors

    Source ↗
  10. 10
    OpenCode Open Source Coding Agent (2026)

    No automatic write access to production files by default, every file modification requires user review and explicit approval

🇨🇳 Qwen10 places · 3 notes

The same question, answered by Qwen alone. The final list above combines all the AIs.

  1. 1
    Claude 3.7 Sonnet

    Completes 88.66% of useful work while shrinking prompt-injection attack success to just 7.31% on AgentDojo.

    Source ↗
  2. 2
    AgentSys (Defended Agent)

    Holds attack success to 0.78% on AgentDojo and 4.25% on ASB while improving benign task utility.

    Source ↗
  3. 3
    Claude Opus 4.6

    Scored a perfect 100% in one targeted security audit, though real-world reversibility of its desktop actions remains debated.

    Source ↗
  4. 4
    Verigrey Defended Agent

    Keeps 93.2% of useful work intact while holding attack success under 1% through action-governance rather than prompt scanning.

    Source ↗
  5. 5
    o3 (Defended)

    Completes 77% of AgentDojo tasks with provable security, compared to 84% for the undefended baseline.

    Source ↗
  6. 6
    Claude 3.5 Sonnet

    Achieves the highest NRP (Normalized Risk-adjusted Performance) on ASB at 43.56, despite baseline attack vulnerabilities.

    Source ↗
  7. 7
    GPT-4o

    Completes 69.07% of tasks but yields to nearly half (47.69%) of prompt injections on AgentDojo without strict guardrails.

    Source ↗
  8. 8
    GPT-4o-mini

    Shows a 49.92% attack success rate on AgentDojo, exposing how smaller models struggle to filter injected tool outputs.

    Source ↗
  9. 9
    Claude 3.5 Sonnet (Red-teamed)

    While baseline attacks only succeed 11% of the time, coordinated red-team prompts breach it 81% of the time.

    Source ↗
  10. 10
    GPT-4-0125

    Early 2024 versions suffered a 56.3% targeted prompt-injection success rate before modern AgentDojo defenses were standardized.

    Source ↗

What Qwen noticed

Action governance outperforms prompt scanning: blocking the result of a tool call keeps attack success under 1% while saving 93.2% of utility.

AgentDojo's targeted prompt-injection attack success rate has fallen from 56.3% on early GPT-4 models to just 7.3% on Claude 3.7 Sonnet, showing rapid security evolution.

The Agent Security Bench introduces the 'Normalized Risk-adjusted Performance' (NRP) metric, proving that refusing malicious prompts doesn't have to mean refusing legitimate work.