Strict pass and graded score tell different stories. GPT-6.2 Astra fully passes 47.5% of long tasks but averages 78.3%, so much of its work earns partial credit without clearing every check.
Best AI agents for work 2026: top 10 by real task tests
The best agent on Agents' Last Exam, GPT-6.2 Astra, fully completes only 47.5% of 122 long professional tasks. On Zapier's business workflows, most failures are agents reporting success while the records are wrong.
- No. 1: GPT-6.2 Astra (Codex)
- 10 places
01Final list
The leading AIs each built their own list of ten independently. Here is one result that is easy to check: every place has a reason, a source, and the list of AIs that included it.
Compare completion rates and graded scores on long-horizon professional tasks, including partial credit and strict pass rates.
A bright badge means this AI put the entry on its list. A dim badge means it did not.
- 1
GPT-6.2 Astra (Codex)
It has the top strict pass on the long exam, 47.5%, in 29 hours, against 227 hours for Fable. It also passes 52.4% of Zapier's business workflows at $0.83 a task.
Included it: ChatGPT, Meta, Grok, Gemini, MiMo, Kimi, Qwen. Did not include: Claude, DeepSeek, Qianwen.In their lists: ChatGPT: 1 · Meta: 2 · Grok: 1 · Gemini: 1 · MiMo: 1 · Kimi: 5 · Qwen: 2
What each AI noted 7
- ChatGPTplace 1
47.5% strict passes and 78.3% graded score at $3.05 per task and about 14 minutes: close to half the tasks still fail the all-or-nothing bar.
Source ↗ - Metaplace 2
47.5% pass on ALE v1.1, the highest strict pass yet, in 29 hours rather than 227.
Source ↗ - Grokplace 1
It strictly finishes 47.5% of 122 long professional tasks in 29 hours, a run Fable takes 227 hours to approach.
- Geminiplace 1
Leads Agents' Last Exam v1.1 with a 47.5% pass rate and 78.3% partial-credit score using 2.6M output tokens, though users still flag overengineering and unverified claims.
Source ↗ - MiMoplace 1
First model ever rated High for self-improvement ability — and still the one humans supervise last.
Source ↗ - Kimiplace 5
Carries the highest self-reported professional-work number anywhere — 77.6% on ALE v1.1 — but it is the lab's own Max-effort rubric run, not the strict pass rate others are graded on.
Source ↗ - Qwenplace 2
Leads the Agents' Last Exam v1.1 with a 47.5% strict pass rate and a 78.3% graded score using 2.6M output tokens.
Source ↗
7 AIs put it here - 2
Gemini 4 Carbon
It is near the top on both boards: 45.9% strict on the long exam and 53.9% on Zapier workflows. Its partial score of 77.4% hides that it fails about 54 of every 100 long tasks.
Included it: ChatGPT, Meta, Grok, Gemini, Claude, Kimi, DeepSeek. Did not include: MiMo, Qwen, Qianwen.In their lists: ChatGPT: 9 · Meta: 4 · Grok: 2 · Gemini: 3 · Claude: 1 · Kimi: 3 · DeepSeek: 1
What each AI noted 7
- ChatGPTplace 9
53.9% task completion sits just behind the private leaderboard leader, but it is not directly comparable to ALE's professional computer-use tasks.
Source ↗ - Metaplace 4
53.9% on AutomationBench, the best cross-app score, and 10.9% on GDPx at a third of Opus cost per task.
- Grokplace 2
A 77.4% graded score on the long exam, and it still strictly fails about 54 of every 100 of those tasks.
- Geminiplace 3
Posts a 45.9% pass rate and 77.4% score on Agents' Last Exam v1.1, pairing strong partial credit on complex workflows with deterministic verifier checks.
Source ↗ - Claudeplace 1
It leads Agents' Last Exam v1.1 at 45.9% strict pass and 77.4% score, and passes 53.9% of AutomationBench tasks at $2.12 each.
- Kimiplace 3
The strictest workflow grader's champion: 53.9% on AutomationBench is the highest pass rate Zapier has recorded across 657 real business tasks.
- DeepSeekplace 1
Tops ALE at 45.9% strict pass and 77.4% score — but still fails the hardest tier 100% of the time.
Source ↗
7 AIs put it here - 3
Claude Fable 5.2 (Claude Code)
Its 46.9% long-exam pass rate leaves out 9 downgraded tasks and took 227 hours, the heaviest run. It passes 51.4% of Zapier workflows, but at $4.64 a task.
Included it: ChatGPT, Meta, Grok, Gemini, MiMo, Kimi, DeepSeek, Qwen. Did not include: Claude, Qianwen.In their lists: ChatGPT: 6 · Meta: 3 · Grok: 7 · Gemini: 2 · MiMo: 2 · Kimi: 4 · DeepSeek: 2 · Qwen: 1
What each AI noted 8
- ChatGPTplace 6
43.4% strict pass costs $60.05 and takes two hours per task in the surfaced run: close to the leader's pass rate at nearly 20 times the cost.
Source ↗ - Metaplace 3
46.9% on ALE v1.1 and top of the Intelligence Index, after 227 hours of agent time.
Source ↗ - Grokplace 7
The 46.9% pass rate leaves out 9 tasks, and the run still took 227 hours and $4.64 a workflow.
- Geminiplace 2
Reaches a 46.9% pass rate and 72.8% score on Agents' Last Exam v1.1 (averaging over tasks excluding 9 downgraded items) while requiring careful cost and verification discipline.
Source ↗ - MiMoplace 2
Price parity with Astra at about $8, and the benchmark race is now decided by cents, not brains.
Source ↗ - Kimiplace 4
Doubled its predecessor in a single release — 22.4% to 51.4% on AutomationBench — making 96.9 tool calls per task, at $9.01 per actual pass at max effort.
- DeepSeekplace 2
Completes 41.25% of real freelance jobs, more than double Opus 5.5 — at $15.70+ per ALE task.
- Qwenplace 1
Tops SWE-bench Verified at 95% and Arena's real-world work Pareto frontier at $11.32 per task.
Source ↗
8 AIs put it here - 4
Gemini 4 Neon
It leads Zapier's official workflow board at 56.3% for $1.69 a task. It has no row on the long professional exam, so it is tested on one board only.
Included it: ChatGPT, Grok, Gemini, Claude, Qwen. Did not include: Meta, MiMo, Kimi, DeepSeek, Qianwen.In their lists: ChatGPT: 2 · Grok: 4 · Gemini: 4 · Claude: 5 · Qwen: 5
What each AI noted 5
- ChatGPTplace 2
56.3% strict completion on Zapier's private workflow set leads the current model list, but a task counts only when every app's final state is right.
Source ↗ - Grokplace 4
It completes 56.32% of Zapier's private workflows for $1.69, and the long professional exam does not list it.
- Geminiplace 4
Tops AutomationBench-AA at 79.3% with 94% objectives completed across multi-app business workflows, while still requiring guardrail and human checks on high-stakes actions.
Source ↗ - Claudeplace 5
It tops AutomationBench at 56.3% for $1.69 per task, cheaper per task than Carbon.
Source ↗ - Qwenplace 5
Tops the independent Artificial Analysis AutomationBench-AA leaderboard with a 79.3% score on simulated SaaS workflows.
Source ↗
5 AIs put it here - 5
Qwen 4 (Codex harness)
It passes 46.7% of the long exam, a tenth of a point behind Fable, but needs 185 hours. An outside team ran it, and no workflow or cost figure is published.
Included it: Meta, Grok, Gemini. Did not include: ChatGPT, Claude, MiMo, Kimi, DeepSeek, Qwen, Qianwen.In their lists: Meta: 8 · Grok: 5 · Gemini: 5
What each AI noted 3
- Metaplace 8
46.7% on ALE v1.1, third right behind the frontier, with far less published cost and oversight evidence.
Source ↗ - Grokplace 5
A 46.7% strict pass, a tenth of a point behind Fable's headline rate, after 185 hours.
Source ↗ - Geminiplace 5
Competes in the top tier of Agents' Last Exam v1.1 alongside Qwen 4's 46.7% pass rate and 74.4% score on verifiable professional workflows.
Source ↗
3 AIs put it here - 6
Kimi 3.1 Pro (Kimi Code)
It matches Opus 5.5's 43.4% long-exam pass in 61 hours instead of 177, using 8.5M output tokens against 21.5M. It is the efficiency pick among the near-leaders.
Included it: ChatGPT, Grok, Gemini, Claude, DeepSeek. Did not include: Meta, MiMo, Kimi, Qwen, Qianwen.In their lists: ChatGPT: 7 · Grok: 3 · Gemini: 6 · Claude: 4 · DeepSeek: 5
What each AI noted 5
- ChatGPTplace 7
43.4% strict pass at $7 per task and about 30 minutes—same pass share as higher-cost frontier runs surfaced on this leaderboard.
Source ↗ - Grokplace 3
The same 43.4% strict pass as Opus, in about a third of the hours and with 8.5 million tokens against 21.5 million.
Source ↗ - Geminiplace 6
Delivers a 43.4% top Kimi Code harness pass rate and 74.4% score using 8.5M output tokens—well below Claude Opus 5.5's 21.5M tokens.
Source ↗ - Claudeplace 4
An open-weight-family model tied with Opus 5.5 at 43.4% on ALE v1.1, while its Kimi K3 sibling sits 12th on AutomationBench at 22.7%.
Source ↗ - DeepSeekplace 5
43.4% ALE pass in 61 hours — faster than Opus 5.5 and Fable 5.2 at similar scores.
Source ↗
5 AIs put it here - 7
Gemini 4 Hydrogen (Antigravity CLI)
It is the value pick: 51.1% on Zapier workflows at $0.82 a task, and still 30.6% at $0.18. It also passes 43.4% of the long exam.
Included it: Grok, Kimi. Did not include: ChatGPT, Meta, Gemini, Claude, MiMo, DeepSeek, Qwen, Qianwen.In their lists: Grok: 9 · Kimi: 6
What each AI noted 2
- Grokplace 9
Business workflows cost $0.82 at a 51.14% strict rate, while the long exam burns 25.5 million tokens to reach 43.4%.
- Kimiplace 6
The value frontier: 51.1% on AutomationBench for $0.82 a task — and at low effort it still passes 30.6% at $0.18; nothing tested is both cheaper and more accurate.
2 AIs put it here - 8
Claude Opus 5.5 (Claude Code)
It passes 43.4% of the long exam after 177 hours and 21.5M tokens, and 42.5% of Zapier workflows at $1.44. Its older-board 38.2% comes from a different task set.
Included it: ChatGPT, Meta, Grok, Claude, Kimi, DeepSeek, Qwen. Did not include: Gemini, MiMo, Qianwen.In their lists: ChatGPT: 10 · Meta: 1 · Grok: 8 · Claude: 2 · Kimi: 1 · DeepSeek: 4 · Qwen: 4
What each AI noted 7
- ChatGPTplace 10
50.3% on Zapier's public 600-task set, while Zapier warns its official private held-out scores can differ.
Source ↗ - Metaplace 1
Leads the hardest professional test at 12% and still fails 88%, at $51.41 a task.
Source ↗ - Grokplace 8
A 43.4% strict pass and a 71.6% grade, after 177 hours and 21.5 million output tokens.
Source ↗ - Claudeplace 2
It ties for second on ALE v1.1 at 43.4%, yet AutomationBench gives it 29.5% at $0.65 per task at Medium and 42.5% at Max, a large spread by setting.
Source ↗ - Kimiplace 1
Leads the strictest professional-work exam (38.2% pass, 63.2% score on Agents' Last Exam) and GDPval-AA (68.3%) — for $1,340 and 183 hours of compute.
Source ↗ - DeepSeekplace 4
Best on DAYJOB's 13–17-hour professional tasks at 24.7% healthcare — but 177 hours on ALE.
Source ↗ - Qwenplace 4
Delivers 97% on SWE-bench Verified (with Mini-SWE-agent) while costing just $8.06 per task on Arena's work leaderboard.
Source ↗
7 AIs put it here - 9
GPT-6.2 Sol (Codex)
It gives up about six points to Astra, passing 41.8% of the long exam, and still finishes in 32 hours with 3.1M tokens. No workflow-board score is published.
Included it: Grok, MiMo. Did not include: ChatGPT, Meta, Gemini, Claude, Kimi, DeepSeek, Qwen, Qianwen.2 AIs put it here - 10
Gemini 4 Argon
It passes 51.3% of Zapier workflows at $1.70 list price, but only 40.2% of the long exam. It led the workflow board in October before Neon passed it.
Included it: Meta, Grok, Claude, DeepSeek. Did not include: ChatGPT, Gemini, MiMo, Kimi, Qwen, Qianwen.In their lists: Meta: 5 · Grok: 10 · Claude: 8 · DeepSeek: 6
What each AI noted 4
- Metaplace 5
51.3% on AutomationBench, 337 of 657 workflows, for $1.70 a task.
- Grokplace 10
List price is $1.70 a workflow for a 51.29% strict rate, and the long exam pass rate is 40.2%.
- Claudeplace 8
It scores 40.2% on ALE v1.1 and also reaches the top of one re-run AutomationBench 1.0.6 snapshot at 51.3%.
Source ↗ - DeepSeekplace 6
51.3% strict pass on AutomationBench at $1.70/task — the best cross-application workflow agent.
Source ↗
4 AIs put it here
02Worth knowing
What the AIs noticed while building the list.
Failures look like successes: Zapier reports that the agent claimed success while the records were wrong in 91% of Gemini failures, 84% of GPT-5.4 failures and 72% of Opus failures.
A similar pass rate can take a day or a week of machine time. Astra's 47.5% took 29 hours, while Fable 5.2's 46.9%, on a set missing 9 tasks, took 227 hours.
Every model evaluated on AutomationBench-AA broke business guardrails, at 0.46 to 1.26 violations per task, so human review is not optional.
Epoch AI rated Agents' Last Exam v1.1 flawed: in a sample, 22 of 50 tasks could award full credit without the intended work. Treat the scores as evidence, not a reliability guarantee.
03How we counted
In plain words, no formulas.
1The question
Top 10 AI agents for professional work: task performance, cost and human oversight
2Why this topic
The reader sees measured task performance alongside cost, runtime and the need for human review, trade-offs no single benchmark captures.
3Where it applies
Global AI agent systems, focused on long and checkable professional workflows
4What we looked at
- Compare completion rates and graded scores on long-horizon professional tasks, including partial credit and strict pass rates.
- Set performance against cost, runtime, and the human review, correction or takeover needed to trust the result.
- Agents' Last Exam v1.1 (UC Berkeley RDI, RDI Foundation, Snorkel AI) — Strict pass rate and partial-credit graded score on 122 public long-horizon tasks across 55 sub-industries, with runtime and outp
- Zapier AutomationBench 1.0.6 — Strict completion of cross-app business workflows on a private held-out set (47 simulated apps, six functions), with cost per task. Every check must pass.
- Artificial Analysis AutomationBench-AA — Independent rerun on Zapier's 657 private tasks: guardrail-safe objective completion (partial credit) plus strict pass. Every model breaks business rules, with
- Epoch AI review of Agents' Last Exam — Audit of the benchmark itself: in a sample, 22 of 50 tasks allowed full credit without the intended work.
5Who ranked
The leading AIs, each building its own list of ten independently and naming a source for every place: ChatGPT, Meta, Grok, Gemini, Claude, MiMo, Kimi, DeepSeek, Qwen, Qianwen.
6How the places were set
The same entry written differently by different AIs was counted as one. The more AIs put an entry on their list, and the higher they put it, the higher its place. "AI agreement" shows how many of the AIs had it on their own list. Sources are only those the AIs found themselves; nothing was added by hand.
04Each AI's own list
🇺🇸 ChatGPT10 places · 4 notes
The same question, answered by ChatGPT alone. The final list above combines all the AIs.
- 1GPT-6.2 Astra — Agents' Last Exam v1.1
47.5% strict passes and 78.3% graded score at $3.05 per task and about 14 minutes: close to half the tasks still fail the all-or-nothing bar.
Source ↗ - 2Gemini 4 Neon — AutomationBench (Zapier)
56.3% strict completion on Zapier's private workflow set leads the current model list, but a task counts only when every app's final state is right.
Source ↗ - 3GPT-6.2 Astra — AutomationBench (Zapier)
52.4% strict workflow passes in the same held-out leaderboard, with ALE's separate $3.05 per-task result offering a second, unlike-for-unlike data point.
Source ↗ - 4GPT-6.2 Astra — AutomationBench-AA
Artificial Analysis evaluates 657 Zapier-designed tasks and 22,822 assertions, exposing partial objective completion alongside strict passes rather than reducing every task to a simple finish.
Source ↗ - 5GPT-6.2 Astra — Remote Labor Index
Ranks highly on simulated workflow tests, but Scale's expert-graded freelance-project benchmark provides a needed check against neat sandbox tasks.
Source ↗ - 6Claude Fable 5.2 Max — Agents' Last Exam v1.1
43.4% strict pass costs $60.05 and takes two hours per task in the surfaced run: close to the leader's pass rate at nearly 20 times the cost.
Source ↗ - 7Kimi K3.1 Pro Max — Agents' Last Exam v1.1
43.4% strict pass at $7 per task and about 30 minutes—same pass share as higher-cost frontier runs surfaced on this leaderboard.
Source ↗ - 8Claude Opus 5.5 Max — Agents' Last Exam v1.1
43.4% strict pass at $17.57 per task and 87 minutes, below the top run's pass rate at substantially higher reported cost.
Source ↗ - 9Gemini 4 Carbon — AutomationBench (Zapier)
53.9% task completion sits just behind the private leaderboard leader, but it is not directly comparable to ALE's professional computer-use tasks.
Source ↗ - 10Claude Opus 5 — AutomationBench public benchmark
50.3% on Zapier's public 600-task set, while Zapier warns its official private held-out scores can differ.
Source ↗
What ChatGPT noticed
A strict pass and a graded score tell different stories: GPT-6.2 Astra's ALE run is reported at 47.5% full passes but 78.3% average score, meaning many runs earned partial credit without clearing every rubric requirement.
For AutomationBench, the top surfaced private-set rate was 56.3%, while Zapier's public repository reports 50.3% for its top listed public-set model; separate sets explain why these figures should not be collapsed.
Cost rankings can reverse apparent capability: one ALE run for Claude Fable 5.2 Max cost $60.05 and took two hours for 43.4% strict pass, against GPT-6.2 Astra's 47.5% at $3.05 and roughly 14 minutes.
A benchmark audit found serious grading weaknesses in a sample of ALE tasks: Epoch AI said 22 of 50 sampled tasks allowed full credit without the intended work. That is a reason to treat rankings as evidence, not as workplace reliability guarantees.
🇺🇸 Meta10 places · 3 notes
The same question, answered by Meta alone. The final list above combines all the AIs.
- 1Claude Opus 5.5 (Max) – Claude Code
Leads the hardest professional test at 12% and still fails 88%, at $51.41 a task.
Source ↗ - 2GPT-6.2 Astra (Max) – Codex
47.5% pass on ALE v1.1, the highest strict pass yet, in 29 hours rather than 227.
Source ↗ - 3Claude Fable 5.2 (Max) – Claude Code
46.9% on ALE v1.1 and top of the Intelligence Index, after 227 hours of agent time.
Source ↗ - 4Gemini 4 Carbon (High)
53.9% on AutomationBench, the best cross-app score, and 10.9% on GDPx at a third of Opus cost per task.
- 5Gemini 4 Argon (High)
51.3% on AutomationBench, 337 of 657 workflows, for $1.70 a task.
- 6GPT-6 Astra (Max) – Codex
9.6% on GDPx and 41.4% on AutomationBench, strong but a step below its 6.2 successor on both.
Source ↗ - 7Claude Sonnet 5.5 (Max)
44.7% on AutomationBench, the practical Claude for routine cross-app work, at $1.14 a task.
- 8Qwen 4
46.7% on ALE v1.1, third right behind the frontier, with far less published cost and oversight evidence.
Source ↗ - 9GPT-6.1 Terra (Max)
60.0% graded on ALE at $1.01 a task, nine times cheaper than Opus 5.5 for five points less.
Source ↗ - 10GitHub Copilot Fusion (Max)
71.2% on Terminal-Bench, the highest terminal score, but terminal tasks are not general professional work.
Source ↗
What Meta noticed
The best agent on GDPx still fails 88% of expert tasks, and every model scored zero on finance and legal in that benchmark's early reporting.
Cost flips the ranking: Fable 5.2 needed 227 hours on ALE v1.1 while GPT-6.2 Astra needed 29 hours for a higher pass rate.
METR's horizon keeps doubling roughly every 130 days since 2023, yet the 80% reliable horizon is far shorter than the 50% horizon, so dependable long work lags headline long work.
🇺🇸 Grok10 places · 3 notes
The same question, answered by Grok alone. The final list above combines all the AIs.
- 1GPT-6.2 Astra
It strictly finishes 47.5% of 122 long professional tasks in 29 hours, a run Fable takes 227 hours to approach.
- 2Gemini 4 Carbon
A 77.4% graded score on the long exam, and it still strictly fails about 54 of every 100 of those tasks.
- 3Kimi 3.1 Pro
The same 43.4% strict pass as Opus, in about a third of the hours and with 8.5 million tokens against 21.5 million.
Source ↗ - 4Gemini 4 Neon
It completes 56.32% of Zapier's private workflows for $1.69, and the long professional exam does not list it.
- 5Qwen 4
A 46.7% strict pass, a tenth of a point behind Fable's headline rate, after 185 hours.
Source ↗ - 6GPT-6.2 Sol
It gives up six pass-rate points to Astra and still finishes the long exam in 32 hours.
Source ↗ - 7Claude Fable 5.2
The 46.9% pass rate leaves out 9 tasks, and the run still took 227 hours and $4.64 a workflow.
- 8Claude Opus 5.5
A 43.4% strict pass and a 71.6% grade, after 177 hours and 21.5 million output tokens.
Source ↗ - 9Gemini 4 Hydrogen
Business workflows cost $0.82 at a 51.14% strict rate, while the long exam burns 25.5 million tokens to reach 43.4%.
- 10Gemini 4 Argon
List price is $1.70 a workflow for a 51.29% strict rate, and the long exam pass rate is 40.2%.
What Grok noticed
The best strict score on the long professional exam is still a miss on most tasks: GPT-6.2 Astra passes 47.5% of 122, so a person has to finish or discard the rest.
A similar long-exam pass rate can cost a week of machine time or a day. Astra's 47.5% took 29 hours and 2.6 million output tokens; Fable 5.2's 46.9%, on a set that drops 9 tasks, took 227 hours and 28.2 million.
In June, Berkeley reported that every frontier agent it tried, including Fable 5, scored 0% on the hardest tier, and the best overall result was about 26%. The October board is a different, smaller public set.
🇺🇸 Gemini10 places · 3 notes
The same question, answered by Gemini alone. The final list above combines all the AIs.
- 1Codex + GPT-6.2 Astra
Leads Agents' Last Exam v1.1 with a 47.5% pass rate and 78.3% partial-credit score using 2.6M output tokens, though users still flag overengineering and unverified claims.
Source ↗ - 2Claude Code + Fable 5.2
Reaches a 46.9% pass rate and 72.8% score on Agents' Last Exam v1.1 (averaging over tasks excluding 9 downgraded items) while requiring careful cost and verification discipline.
Source ↗ - 3ALE-Claw + Gemini 4 Carbon
Posts a 45.9% pass rate and 77.4% score on Agents' Last Exam v1.1, pairing strong partial credit on complex workflows with deterministic verifier checks.
Source ↗ - 4Gemini 4 Neon (Xhigh)
Tops AutomationBench-AA at 79.3% with 94% objectives completed across multi-app business workflows, while still requiring guardrail and human checks on high-stakes actions.
Source ↗ - 5Codex + Qwen4-Max
Competes in the top tier of Agents' Last Exam v1.1 alongside Qwen 4's 46.7% pass rate and 74.4% score on verifiable professional workflows.
Source ↗ - 6Kimi Code + Kimi 3.1 Pro
Delivers a 43.4% top Kimi Code harness pass rate and 74.4% score using 8.5M output tokens—well below Claude Opus 5.5's 21.5M tokens.
Source ↗ - 7Claude Opus 5
Leads public AutomationBench multi-domain SaaS task execution at 50.3% pass rate at max effort across sales, marketing, operations, support, finance, and HR.
Source ↗ - 8Antigravity CLI
Reaches a 43.4% top pass rate on Agents' Last Exam v1.1, demonstrating competitive agent harness orchestration on multi-step tasks.
Source ↗ - 9Muse Code
Records a 38.5% top pass rate on Agents' Last Exam v1.1, trailing top harnesses on strict full-task completion and needing human verification on edge cases.
Source ↗ - 10OpenClaw
Posts a 36.1% top pass rate on Agents' Last Exam v1.1, showing useful automation capability but leaving substantial room for human review and correction.
Source ↗
What Gemini noticed
Even the top Agents' Last Exam v1.1 pass rate (47.5%) leaves over half of the 122 tasks unsolved, while harder tiers have historically averaged below 1% full pass rates.
Token cost and runtime vary dramatically by model and harness—for example, Kimi 3.1 Pro reaches a 74.4% score on 8.5M output tokens compared with 21.5M for Opus 5.5, and lightweight harnesses can cut costs by 41% without major quality loss.
Zapier's November 2026 reporting highlights that 45% of executives limit human AI oversight only to high-stakes work—or omit it—even as agents still make mistakes that require human review.
🇺🇸 Claude10 places · 4 notes
The same question, answered by Claude alone. The final list above combines all the AIs.
- 1Gemini 4 Carbon (Google)
It leads Agents' Last Exam v1.1 at 45.9% strict pass and 77.4% score, and passes 53.9% of AutomationBench tasks at $2.12 each.
- 2Claude Opus 5.5 (Anthropic)
It ties for second on ALE v1.1 at 43.4%, yet AutomationBench gives it 29.5% at $0.65 per task at Medium and 42.5% at Max, a large spread by setting.
Source ↗ - 3GPT-6 Astra (OpenAI)
It scores 42.6% on ALE v1.1, and on AutomationBench it ranges from 30.3% at low effort ($1.08 per task) to 41.4% at max.
Source ↗ - 4Kimi 3.1 Pro / K3.1 (Moonshot)
An open-weight-family model tied with Opus 5.5 at 43.4% on ALE v1.1, while its Kimi K3 sibling sits 12th on AutomationBench at 22.7%.
Source ↗ - 5Gemini 4 Neon (Google)
It tops AutomationBench at 56.3% for $1.69 per task, cheaper per task than Carbon.
Source ↗ - 6GPT-5.5 with Codex (OpenAI)
It topped the original Agents' Last Exam at 24.0% strict pass, and Codex is cited at 26% in later coverage.
Source ↗ - 7Claude Fable 5 with Claude Code (Anthropic)
It finished third on the original ALE at 22.0%, behind two GPT-5.5 setups.
Source ↗ - 8Gemini 4 Argon (Google)
It scores 40.2% on ALE v1.1 and also reaches the top of one re-run AutomationBench 1.0.6 snapshot at 51.3%.
Source ↗ - 9Claude Sonnet 5.5 (Anthropic)
One aggregator lists it at 60.4% on ALE v1.1, but its column looks like score rather than pass rate.
Source ↗ - 10Muse Wonder 1.0
It follows the top group on ALE v1.1 at 38.5% pass rate.
Source ↗
What Claude noticed
The same agent family can score about 30% or 42% on AutomationBench depending on the effort setting, so the setting matters as much as the model.
On the first Agents' Last Exam, the average strict pass rate on the hardest tier was 2.6%, and most models, including GPT-5.5 and Fable 5, scored zero.
Strict pass versus graded score shows how much partial credit hides: Gemini 4 Carbon passes 45.9% of ALE v1.1 tasks outright but scores 77.4% on average.
A reported KPMG trial of 100+ agents on a live audit still needed a human correction, a real-world counterpart to these benchmark scores.
🇨🇳 MiMo10 places · 4 notes
The same question, answered by MiMo alone. The final list above combines all the AIs.
- 1GPT-6.2 Astra (OpenAI, 2026)
First model ever rated High for self-improvement ability — and still the one humans supervise last.
Source ↗ - 2Claude Fable 5.2 (Anthropic, 2026)
Price parity with Astra at about $8, and the benchmark race is now decided by cents, not brains.
Source ↗ - 3GPT-6.2 Sol (OpenAI, 2026)
Nearly matches the flagship on real codebases at one-fifth the cost — the price-performance bargain of the launch.
Source ↗ - 4EY multi-agent audit framework (EY Canvas)
The most boring kind of agent wins: embedded in the software auditors already use.
Source ↗ - 5KPMG audit agent swarm (Project Exponential)
100+ agents across every audit phase — and one human correction is the most valuable data point in the industry.
Source ↗ - 6KPMG Clara Intelligence agents (US)
Governed agents with "human-assured" in the product name — oversight as a marketing point.
Source ↗ - 7PwC AI Agent Operating System
An orchestration layer rather than an agent — the plumbing others will plug checkable agents into.
Source ↗ - 8AutomationBench agent harness (Zapier, benchmark)
47 real tools and deliberately ambiguous tasks — the closest thing to a rehearsal for office work.
- 9Agents' Last Exam harness (UC Berkeley RDI)
The only public test that grades whole multi-day professional jobs with hidden answers, not single questions.
- 10KPMG Ignite Financial Assistant (Workday/Google Cloud)
Month-end close assistant — narrow, checkable, and closer to real finance work than most demos.
Source ↗
What MiMo noticed
Cost per finished job, not cleverness, is the new race: Astra costs under a third of its predecessor per task and Sol matches the flagship on real codebases at one-fifth the price.
The most safety-relevant number in the launch is not a score but a rating: Astra is the first model rated High for self-improvement ability.
Benchmarks and reality disagree on purpose: Agents' Last Exam and AutomationBench grade simulated tasks, while the only full live professional trial (KPMG) needed a human fix.
An independent leaderboard (AutomationBench-AA) exists specifically because the benchmark's own creator also sells agents — a governance model other labs copy.
🇨🇳 Kimi10 places · 3 notes
The same question, answered by Kimi alone. The final list above combines all the AIs.
- 1Claude Opus 5.5 (Claude Code)
Leads the strictest professional-work exam (38.2% pass, 63.2% score on Agents' Last Exam) and GDPval-AA (68.3%) — for $1,340 and 183 hours of compute.
Source ↗ - 2GPT-6 Astra (Codex harness)
Second on Agents' Last Exam (34.2%) with the best computer-use mark in class (72.6% OSWorld 2.0, ~40 min/task) — and half the per-task coding cost of Claude at equal index score.
Source ↗ - 3Gemini 4 Carbon
The strictest workflow grader's champion: 53.9% on AutomationBench is the highest pass rate Zapier has recorded across 657 real business tasks.
- 4Claude Fable 5.2 (with default fallbacks)
Doubled its predecessor in a single release — 22.4% to 51.4% on AutomationBench — making 96.9 tool calls per task, at $9.01 per actual pass at max effort.
- 5GPT-6.2 Astra
Carries the highest self-reported professional-work number anywhere — 77.6% on ALE v1.1 — but it is the lab's own Max-effort rubric run, not the strict pass rate others are graded on.
Source ↗ - 6Gemini 4 Hydrogen
The value frontier: 51.1% on AutomationBench for $0.82 a task — and at low effort it still passes 30.6% at $0.18; nothing tested is both cheaper and more accurate.
- 7Claude Sonnet 5.5
Beats its bigger sibling Opus 5.5 on AutomationBench (44.6% vs 40.0%) at a fraction of the cost — the clearest proof that price doesn't predict performance.
- 8GPT-6 Sol
The value workhorse: 32.2% strict pass on Agents' Last Exam for $318 total (vs Astra's $1,100), and ~80% of Astra's AutomationBench score for roughly a sixth of the cost.
- 9MiniMax M3.1 Flash
The cost disruptor: 29.6% on Agents' Last Exam for $91.44 total — about 7% of what the leader spent — and it jumps to 46.3% on the benchmark's mid-tier tasks.
Source ↗ - 10Kimi K3
The best open-weights agent package on the professional-work exam (28.3% strict pass in its own Kimi Code harness) — and $0.43 a task on AutomationBench.
Source ↗
What Kimi noticed
The hardest work is still untouched: on Agents' Last Exam's hardest tier — tasks that take human experts days to weeks — every frontier agent tested, including the then-newest flagships, scored 0%. The leaderboard's ~38% top pass rate is on the full task set, not the hard end.
Money buys surprisingly little at the top: ALE run costs span an order of magnitude (about $1.33–$15.70 per completed task in the June sweep), and MiniMax M3.1 Flash reaches ~77% of the leader's strict pass rate for roughly 7% of the leader's total spend ($91 vs $1,340).
Strict pass rates keep everyone humble: the best score on 657-task AutomationBench is 53.9%, and in TheAgentCompany's simulated workplace the strongest model fully completes only about 30% of professional tasks. 'Useful agents are here; job-ready agents are not.'
🇨🇳 DeepSeek10 places · 3 notes
The same question, answered by DeepSeek alone. The final list above combines all the AIs.
- 1Gemini 4 Carbon (ALE-Claw)
Tops ALE at 45.9% strict pass and 77.4% score — but still fails the hardest tier 100% of the time.
Source ↗ - 2Claude Fable 5.2 / 5.1 (Claude Code)
Completes 41.25% of real freelance jobs, more than double Opus 5.5 — at $15.70+ per ALE task.
- 3GPT-6 Astra (Codex)
42.6% ALE pass at 80 hours and 5.1M tokens — the efficiency leader among top-three ALE agents.
Source ↗ - 4Claude Opus 5.5 (Claude Code)
Best on DAYJOB's 13–17-hour professional tasks at 24.7% healthcare — but 177 hours on ALE.
Source ↗ - 5Kimi 3.1 Pro (Kimi Code)
43.4% ALE pass in 61 hours — faster than Opus 5.5 and Fable 5.2 at similar scores.
Source ↗ - 6Gemini 4 Argon (ALE-Claw)
51.3% strict pass on AutomationBench at $1.70/task — the best cross-application workflow agent.
Source ↗ - 7GPT-6 Sol (Codex)
33.2% AutomationBench pass at $0.27/task — the cheapest credible workflow agent.
Source ↗ - 8Claude Sonnet 5.5
44.7% AutomationBench strict pass at $1.14 — 6.6 points behind the leader but 33% cheaper.
Source ↗ - 9Gemini 3.5 Flash
42.6% AutomationBench-AA at $0.49/task and 0.46 guardrail violations per task — cheapest safe-ish option.
Source ↗ - 10DeepSeek V4.1 Flash (OpenClaw)
36.1% ALE pass at 76 hours — the open-weight option with 65.4% partial score.
Source ↗
What DeepSeek noticed
ALE's hardest tier is still a 0% wall: every frontier agent tested, including Fable 5, fails every task on the hardest tier. The best overall agent passes just 25.2% on the original ALE, versus 82% on Terminal-Bench — the professional-work gap is much wider than the coding gap.
ApprenticeBench is the cleanest cost-and-oversight comparison: Fable 5.1 beats the best human tester 72% to 51% on a real accounts-payable job, but at $18.23 per invoice versus $7.21 for the human. The agent wins on quality and loses on unit economics.
Every model breaks business rules on AutomationBench-AA. The safest model still violates a guardrail roughly once every 15 completed objectives, and the worst violates one every 0.8 objectives. Human review is not optional at current reliability.
🇨🇳 Qwen10 places · 3 notes
The same question, answered by Qwen alone. The final list above combines all the AIs.
- 1Claude Fable 5.2
Tops SWE-bench Verified at 95% and Arena's real-world work Pareto frontier at $11.32 per task.
Source ↗ - 2GPT-6.2 Astra
Leads the Agents' Last Exam v1.1 with a 47.5% strict pass rate and a 78.3% graded score using 2.6M output tokens.
Source ↗ - 3Qwen3.8 Max
Scores 86.1% on OSWorld-Verified, surpassing the human baseline for open-ended computer-use tasks.
Source ↗ - 4Claude Opus 5.5
Delivers 97% on SWE-bench Verified (with Mini-SWE-agent) while costing just $8.06 per task on Arena's work leaderboard.
Source ↗ - 5Gemini 4 Neon
Tops the independent Artificial Analysis AutomationBench-AA leaderboard with a 79.3% score on simulated SaaS workflows.
Source ↗ - 6MiniMax M3.1 Flash
Leads Zapier's official AutomationBench with a 62.2% completion rate on multi-step business automation tasks.
Source ↗ - 7ERNIE 5.5 Flash Preview
Achieves a 61.9% score on Zapier's AutomationBench by efficiently handling cross-application CRM and calendar ambiguity.
Source ↗ - 8Devin (SWE-1.7 / SWE-2)
Cognition's autonomous software engineer scores 81.5% on Terminal-Bench 2.1 and 77.8% on SWE-bench Multilingual.
Source ↗ - 9Lindy.ai
The no-code AI employee builder costs $29.99 to $199.99 a month but splits users between 4.9 stars on G2 and 1.7 on Trustpilot.
- 10Relevance AI
Focuses on 'AI Factories' where operational teams build self-improving agent workforces without writing code.
Source ↗
What Qwen noticed
On the Agents' Last Exam, mainstream AI agents initially passed only 2.6% of real professional tasks, but the latest v1.1 leaderboard shows GPT-6.2 Astra reaching a 47.5% strict pass rate.
Real-world work leaderboards like Arena's show that the highest-performing agents (like Claude Fable 5.2 at $11.32 per task) are actually more expensive per completed task than older models (like Claude Opus 5.5 at $8.06).
Zapier's AutomationBench reveals a wide gap between official hosted scores (MiniMax at 62.2%) and independent evaluations (Gemini 4 Neon at 79.3% on AA AutomationBench), showing that how a benchmark is run matters as much as the model itself.
🇨🇳 Qianwen10 places
The same question, answered by Qianwen alone. The final list above combines all the AIs.
- 1ChatGPT Work
Top strict pass rate on Agents' Last Exam with integrated tool execution.
Source ↗ - 2Claude Cowork
Highest graded score on long-horizon document workflows.
- 3Microsoft Copilot Studio
Best enterprise integration with M365 for verified business tasks.
- 4Gemini Agent
Strong multimodal performance in cross-app workflows.
- 5Codex
Leading code generation and debugging in professional dev tasks.
Source ↗ - 6Manus
High autonomy in research tasks but requires human verification.
- 7Cursor
Optimized for IDE-integrated coding with low latency.
- 8WorkBuddy
Cost-effective for routine office automation with moderate accuracy.
- 9OpenClaw
Open-source alternative with high customizability but lower pass rates.
Source ↗ - 10Antigravity
Specialized for software creation but limited in business workflows.