Poly Rating · from the trend of the day

Best AI Model for Agents (Open-Weight, October 2026)

Mistral Large 4 ranks #6 of 75 on Harvey's legal-agent test with just 15.83% and #32 of 44 on the broader Vals Index, where DeepSeek V4.1 Flash reaches 61.26% for $0.66 a test.

  • No. 1: Mistral Large 4
  • 10 places

01Final list

The leading AIs each built their own list of ten independently. Here is one result that is easy to check: every place has a reason, a source, and the list of AIs that included it.

Order: from most to least AI agreement.

A bright badge means this AI put the entry on its list. A dim badge means it did not.

  1. 1

    Mistral Large 4

    The smallest gap on cyber: Mistral claims about 82% and independent tests measured 79–81.7%. Vals measures 26.26% on Terminal-Bench 4.0 and 15.83% on Harvey's legal-agent test.

    Included it: Claude, ChatGPT, Meta, Gemini, Wenxin, DeepSeek, Qwen. Did not include: Nova.

    In their lists: Claude: 2 · ChatGPT: 10 · Meta: 5 · Gemini: 5 · Wenxin: 3 · DeepSeek: 3 · Qwen: 3

    What each AI noted 7
    • Claudeplace 2

      Its cyber claim has the smallest visible gap: Mistral says ~82% on CyberGym-E2E, an independent index measured 81.7%.

      Source ↗
    • ChatGPTplace 10

      Apache 2.0 offers an operational advantage when a business needs to host customer agents without Large 4's Model-as-a-Service exception.

      Source ↗
    • Metaplace 5

      Vendor 93% Cybench, 82% vulnerability patching, 61.7% DeepSWE v1.1 – independent 41% Terminal-Bench 4.0 and 48% Coding Agent Index using 47% fewer tokens

      Source ↗
    • Geminiplace 5

      Claims a breathtaking 82% on CyberGym-E2E and 61.7% on DeepSWE v1.1, but drops to 15.83% on Harvey's independent Legal Agent Benchmark.

      Source ↗
    • Wenxinplace 3

      123B agent scores look great on paper — independent tests show a 20-point gap

      Source ↗
    • DeepSeekplace 3

      Mistral’s launch claims 82% CyberGym-E2E, 93% Cybench and 61.7% DeepSWE — but independent Cyber Index testing puts it around 51%, a ~31-point trust gap that is the defining number for security-agent buyers.

      Source ↗
    • Qwenplace 3

      Mistral claims 82% on CyberGym and 61.7% on DeepSWE, but independent testers measure 81.7% and 60%, showing a rare 'honest' trust gap.

      Source ↗
    7 AIs put it here
  2. 2

    DeepSeek V4.1 Flash

    Best open model on the Vals Index at 61.26% for $0.66 a test, and best released model on FrontierCode at 28%. Its independent Terminal-Bench 4.0 results disagree: 19.7% vs 31.8%.

    Included it: Claude, ChatGPT, Meta, DeepSeek, Qwen. Did not include: Gemini, Nova, Wenxin.

    In their lists: Claude: 3 · ChatGPT: 1 · Meta: 1 · DeepSeek: 1 · Qwen: 2

    What each AI noted 5
    • Claudeplace 3

      Mistral's chart left it out, and Reflection's table puts it above Large 4 on DeepSWE (74.2% vs 61.7%).

      Source ↗
    • ChatGPTplace 1

      Its 28% independent FrontierCode completion leads released open weights, and one measured provider charges $0.37 per composite intelligence-test attempt.

      Source ↗
    • Metaplace 1

      Vals Index open leader 61.26% at $0.66 per test vs Sonnet 5.5 at $18.70 – 28x cheaper, 90.6 Terminal-Bench 2.1 vendor but 31.8 independent

      Source ↗
    • DeepSeekplace 1

      A 61.26% independent Vals Index score at $0.66 per test — but its vendor-reported 74.2 DeepSWE collapses to 19.7% on Vals’ independent Terminal-Bench 4.0, one of the widest trust gaps in the field.

    • Qwenplace 2

      The MIT-licensed workhorse completes complex multi-file coding agents at a fraction of the cost of US frontier models.

    5 AIs put it here
  3. 3

    Xiaomi MiMo-V2.6-Pro

    Scores 57.81–59.5% on the Vals Index at $0.13–$0.24 per test attempt. Its 89.9 on Terminal-Bench 2.1 and 71.9 on DeepSWE are Xiaomi's own numbers.

    Included it: Claude, ChatGPT, Meta, DeepSeek, Qwen. Did not include: Gemini, Nova, Wenxin.

    In their lists: Claude: 7 · ChatGPT: 3 · Meta: 2 · DeepSeek: 5 · Qwen: 8

    What each AI noted 5
    • Claudeplace 7

      A Vals write-up headlines it at about 100 times cheaper than Claude Opus 5.5.

    • ChatGPTplace 3

      Combines 57.81% Vals professional accuracy with a reported $0.24 cost per intelligence-test attempt at GMI.

      Source ↗
    • Metaplace 2

      89.9 Terminal-Bench 2.1 just above Claude Opus 5 at 89.1, 71.9 DeepSWE v1.1, 94% CyberGym, $0.13 per Intelligence task

      Source ↗
    • DeepSeekplace 5

      A 59.5% independent Vals score with full MIT licensing and $0.435/$0.87 per million tokens — the best cost-per-finished-task option among genuinely permissive models.

      Source ↗
    • Qwenplace 8

      A specialized reasoning agent that beats general models on math-heavy coding tasks without needing a 100B parameter footprint.

      Source ↗
    5 AIs put it here
  4. 4

    GLM-5.3 (Z.ai)

    Scores 48–49 on Artificial Analysis' Coding Agent Index and 40% on Terminal-Bench 4.0. Its 84.5% CyberGym score comes from Z.ai, which published its method, but no one has replicated it yet.

    Included it: Claude, ChatGPT, Meta, DeepSeek. Did not include: Gemini, Nova, Wenxin, Qwen.

    In their lists: Claude: 1 · ChatGPT: 2 · Meta: 3 · DeepSeek: 2

    What each AI noted 4
    • Claudeplace 1

      Z.ai published its test method (Claude Code 2.1.207, pass@1 over 1,507 CyberGym tasks) so anyone can contest its 84.5, but nobody independent has yet.

    • ChatGPTplace 2

      Matches DeepSeek's intelligence index at 32 but costs $2.53 per intelligence-test attempt at one independently measured provider.

      Source ↗
    • Metaplace 3

      84.5% CyberGym white-box – technically ahead of Claude Mythos 5 at 83.8%, DeepSWE 66.9% vs 46.2 prior, 40% Terminal-Bench 4.0

      Source ↗
    • DeepSeekplace 2

      Vals’ coding category gives it 64.3 — the best open-weight coding-agent score — and its independent numbers hold up better than most, but its license dispute matters for anyone hosting agents for customers.

      Source ↗
    4 AIs put it here
  5. 5

    Kimi K3 (Moonshot AI)

    About 57–58% on the Vals Index. Its 88.3 on Terminal-Bench 2.1 is Moonshot's own figure, Meta's table lists 85.0, and the weights need 1.4 TB.

    Included it: Claude, ChatGPT, Meta, DeepSeek. Did not include: Gemini, Nova, Wenxin, Qwen.

    In their lists: Claude: 4 · ChatGPT: 8 · Meta: 6 · DeepSeek: 4

    What each AI noted 4
    • Claudeplace 4

      Its 88.3 on Terminal-Bench 2.1 is Moonshot's own, while Meta's table lists 85.0, and the weights take 1.4 TB.

      Source ↗
    • ChatGPTplace 8

      Its strong human-preference showing makes it worth testing as a planning or review agent, but that is not proof of completed tool workflows.

    • Metaplace 6

      Top open on Vals Index at 57%, 1488 Arena, K3.1 Pro scores 67% Terminal-Bench 4.0 in v5 – best open terminal score

      Source ↗
    • DeepSeekplace 4

      A 57.8% independent Vals score and a 1,487 LMArena open-weight Elo, but the $20M revenue trigger and 100M-MAU cap mean agent platforms must budget for a commercial deal, not just GPUs.

      Source ↗
    4 AIs put it here
  6. 6

    Qwen3.8 Max (2.4T-A95B)

    Its vendor Terminal-Bench 2.1 numbers clash (86.6 vs 67.4) and it has no independent agent score. At 2.4T parameters, it carries a $50M gate for service and coding-assistant businesses.

    Included it: Claude, Meta, DeepSeek, Qwen. Did not include: ChatGPT, Gemini, Nova, Wenxin.

    In their lists: Claude: 6 · Meta: 8 · DeepSeek: 6 · Qwen: 5

    What each AI noted 4
    • Claudeplace 6

      The 92.6 GPQA claim is for the Max version, not the downloadable checkpoint.

    • Metaplace 8

      2.4T total 95B active, custom $50M MaaS gate, 68.89% IOI, Apache 27B sibling is clean license fallback

      Source ↗
    • DeepSeekplace 6

      Strong coding-agent numbers (67.4 on Terminal-Bench 2.1 in vendor tests) but a $50M revenue gate for competing MaaS and coding-assistant businesses — the agent platform that hosts it for customers must plan a commercial deal.

      Source ↗
    • Qwenplace 5

      Alibaba's agent orchestrator dominates Asian legal and financial workflows, though its $50M revenue license cap slows down global SaaS adoption.

    4 AIs put it here
  7. 7

    Meta Muse Spark Open

    #1 on Harvey's legal-agent test at 25.00% and #3 of 79 on Vals finance agents, yet only 46.61% on the Vals Index, at $4.08 a test (one source).

    Included it: Claude, Meta, DeepSeek. Did not include: ChatGPT, Gemini, Nova, Wenxin, Qwen.

    In their lists: Claude: 5 · Meta: see list · DeepSeek: 7

    What each AI noted 3
    • Claudeplace 5

      Meta says it beats Kimi K3 on tool use but trails on coding: 76.4 vs 85.0 on Terminal-Bench 2.1.

      Source ↗
    • Meta

      Named differently in its own list: open the full list.

      see list
    • DeepSeekplace 7

      Ranks first on Vals AI’s Harvey Legal Agent Benchmark at 25.00%, but its $20M MaaS-resale trigger means legal-tech platforms need a separate deal before hosting it for clients.

    3 AIs put it here
  8. 8

    GLM-5.3 Flash (Z.ai)

    An MIT-licensed budget agent: 47.22% on the Vals Index at $0.15/$0.50 per million tokens. Its 84.3 on Terminal-Bench 2.1 is the vendor's own figure.

    Included it: Meta, DeepSeek. Did not include: Claude, ChatGPT, Gemini, Nova, Wenxin, Qwen.

    In their lists: Meta: 4 · DeepSeek: 9

    What each AI noted 2
    • Metaplace 4

      MIT, $0.15/$0.50, 84.3 Terminal-Bench 2.1 vendor, best cost to run open agent on single-host

      Source ↗
    • DeepSeekplace 9

      Only a 23 on AA v5, but OpenMDW-1.1 opens weights, training data and recipes — the most transparent long-running-agent model, at $0.60/$2.60 per million tokens.

    2 AIs put it here
  9. 9

    NVIDIA Nemotron 3.5 Bolt 74B

    The lightest option: 74B with 9B active, fitting a 24 GB GPU at 2-bit. Its 50.98 on Terminal-Bench 2.1 is NVIDIA's own figure, with no independent score yet.

    Included it: Claude, Meta. Did not include: ChatGPT, Gemini, Nova, Wenxin, DeepSeek, Qwen.

    In their lists: Claude: 8 · Meta: 10

    What each AI noted 2
    • Claudeplace 8

      The lightest of the group: 74B with 9B active, fitting in 24 GB at 2-bit.

    • Metaplace 10

      Best US hardware fit – Gemma 1452 Arena on 31B, Nemotron 74B total 9B active fits 24GB GPU, Apache 2.0 + NVIDIA open license

      Source ↗
    2 AIs put it here
  10. 10

    DeepSeek V4 Pro

    76% on CyberGym-E2E with zero refusals across 840 adversarial tasks, close to Mistral's 79%, under MIT. Only one source reports this.

    Included it: Meta. Did not include: Claude, ChatGPT, Gemini, Nova, Wenxin, DeepSeek, Qwen.

    In their lists: Meta: 7

    What each AI noted 1
    • Metaplace 7

      MIT, zero refusals on 840 adversarial security tasks, 76% CyberGym-E2E vs 79% for Mistral Large 4, half cost per success

      Source ↗
    1 AIs put it here

More about the top three ↓

02More about the top three

The ranking is above. Here is more about the three places at the top: what they are, why they are here, and what is worth knowing.

The three are not measured the same way: Mistral's place rests on how close its cyber claim is to independent tests, while the other two are judged on Vals Index scores and prices per test, which differ by provider.

1Mistral Large 4

No. 1 — Mistral Large 4. On the lists of 7 of 8 AIs; median place there: 3. The AIs put it first because its cyber claim has the smallest gap to what independent tests measured, even though its other scores are weaker.

Did you know

  • Mistral 7B was released on 27 September 2023 as a BitTorrent magnet link, not a normal download page. Mixtral 8x7B (December 2023) and Mixtral 8x22B (April 2024) were also shared as torrent links on Twitter.

What it is
A large Mistral model that can be used for agents, with open weights promised.
Its edge
Its cyber claim of about 82% is the closest to independent results.
Who it suits
Suits security-minded teams that value honest claims; not those who need low cost or speed.
  • Vals Index score#32 of 44 on Vals Index
Did you know
  • Mistral's first fundraise, in June 2023, was €105 million ($117 million). The Financial Times estimated the company's value at €240 million, and the company was only about two months old.

History
  • On October 6, 2026, Mistral opened a public preview of Mistral Large 4. It said the downloadable weights would follow by the end of October, after more safety work. The article says that safety step was not yet finished.

Keep in mind
  • It thinks a lot before it answers, so it uses many words. One test counted about 200M tokens versus a median of about 81M. First token took about 18.7 seconds on a long prompt.

  • It scored 38.4 on the Artificial Analysis Intelligence Index, behind Kimi K3 and Qwen 3.8 Max. Its hallucination rate was 41.9%. Cheaper models like DeepSeek V4.1 Flash score about the same for much less per task.

Where to go further
  • Open Mistral's Large 4 announcement and read the vendor benchmark list first, starting with the 82% cyber claim. The page says open weights were due by the end of the month, so check whether they are out.

Compare the top three ↓

2DeepSeek V4.1 Flash

No. 2 — DeepSeek V4.1 Flash. On the lists of 5 of 8 AIs; median place there: 1. The AIs ranked it second for the best open-model Vals Index score at a low price per test, despite uneven Terminal-Bench results.

Did you know

  • Despite its "Flash" name, the model has about 552B parameters in total. Only about 8B are active per token while reading a prompt and about 16B while writing the answer, which keeps it cheap to run.

What it is
A cheap, fast DeepSeek model with a huge memory for long texts.
Its edge
It tops the open models on the Vals Index and costs far less per test.
Who it suits
Suits coders who want strong results cheaply; not those who need steady results on every test.
  • Vals Index score61.26% at $0.66 a test
Did you know
  • DeepSeek V4.1 Flash can read about 1 million tokens at once, roughly several long novels. It can also write up to 384K tokens in one answer, though the exact limits vary by provider.

History
  • DeepSeek V4.1 Flash was released on September 10, 2026. Its listing says it "succeeds" V4 Flash and V4 Pro. The listing does not say whether those older models were retired.

Keep in mind
  • The big scores are DeepSeek's own and not independently checked. On Humanity's Last Exam, a hard academic test, it scores 36.8 against 56.3 for Opus 5. It also trails US models on Terminal-Bench 3.0.

Where to go further
  • Open the Vals AI page and look at the Terminal-Bench 4.0 score first. It shows 19.70% on that test, so compare it with the 2.1 result on the same page to see why the numbers differ.

ChatGPT, Meta, DeepSeek put it first. What they saw in it
  • ChatGPT: Its 28% independent FrontierCode completion leads released open weights, and one measured provider charges $0.37 per composite intelligence-test attempt.

  • Meta: Vals Index open leader 61.26% at $0.66 per test vs Sonnet 5.5 at $18.70 – 28x cheaper, 90.6 Terminal-Bench 2.1 vendor but 31.8 independent

  • DeepSeek: A 61.26% independent Vals Index score at $0.66 per test — but its vendor-reported 74.2 DeepSWE collapses to 19.7% on Vals’ independent Terminal-Bench 4.0, one of the widest trust gaps in the field.

Compare the top three ↓

3Xiaomi MiMo-V2.6-Pro

No. 3 — Xiaomi MiMo-V2.6-Pro. On the lists of 5 of 8 AIs; median place there: 5. The AIs placed it third for a solid Vals Index score at a very low price, with its best numbers coming only from Xiaomi.

Did you know

  • MiMo-V2.6-Pro has about 1.02 trillion parameters, but only 42 billion are active per token. Its weights are on Hugging Face under the MIT license, which allows commercial use.

What it is
A huge Xiaomi model with open MIT-licensed weights, built for agent work.
Its edge
It costs the least per test of the three, at $0.13–$0.24.
Who it suits
Suits teams that want a cheap, open model; not those without big GPU servers.
  • Vals Index score57.81–59.5% at $0.13–$0.24
Did you know
  • The release reportedly includes the training code and the reinforcement-learning environments, not only the weights. That is rarer than a plain model download.

History
  • After MiMo-V2.6 came out in 2026, agent users found it sometimes sent the same tool call again and again, looking busy while making no progress. Xiaomi's later MOPD checkpoint is measured against that problem.

Keep in mind
  • Running it yourself is heavy: "You need about 573 GB of weights in GPU memory before any KV cache." The guide says that in practice means 8x H200 cards.

  • The high agent scores are Xiaomi's own, measured in its own harnesses. The guide says "nobody outside Xiaomi can reproduce" the internal ones. It is also wordy: 140M output tokens across the evaluation suite.

Where to go further
  • Read the model card's benchmark table first, then the recommended settings. The scores there are Xiaomi's own, so check them against independent results like Vals before you rely on them.

Compare the top three ↓

The top three side by side

Mistral Large 4DeepSeek V4.1 FlashXiaomi MiMo-V2.6-Pro
In the AIs' lists7 / 85 / 85 / 8
Vals Index score#32 of 44 on Vals Index61.26% at $0.66 a test57.81–59.5% at $0.13–$0.24
Put first by—ChatGPT, Meta, DeepSeek—

Places with the same number of AIs are ordered by their average place in the AIs' own lists.

03Worth knowing

What the AIs noticed while building the list.

The harness moves agent scores as much as the model does: one model scored 45.9% on SWE-Bench Pro with a standard scaffold and 55.4% with its vendor's own agent.

Noticed by:
Included it: Claude, DeepSeek. Did not include: ChatGPT, Meta, Gemini, Nova, Wenxin, Qwen.
Source ↗

Vendors quote Terminal-Bench 2.1 scores of 76–90, while independent Terminal-Bench 4.0 runs land at roughly 20–41%. The versions differ, so the gap is a warning, not a measurement.

Noticed by:
Included it: Claude, Meta, DeepSeek. Did not include: ChatGPT, Gemini, Nova, Wenxin, Qwen.
Source ↗

Mistral Large 4 scores 81.56% on Vibe Code but 26.26% on Terminal-Bench 4.0, and 67.68% on the Vals Legal Index but 15.83% as a legal agent. Knowing the material is not the same as finishing the job.

Noticed by:
Included it: ChatGPT, Gemini. Did not include: Claude, Meta, Nova, Wenxin, DeepSeek, Qwen.
Source ↗

Mistral's cyber claim is the rare near-match between launch number and independent test (~82% vs 81.7%), but part of its lead comes from closed models refusing the tasks.

Noticed by:
Included it: Claude, Meta, Qwen. Did not include: ChatGPT, Gemini, Nova, Wenxin, DeepSeek.
Source ↗

DeepSeek V4.1 Flash costs $0.66 per Vals test against $18.70 for Claude Sonnet 5.5, about 28 times cheaper for roughly 9 points less accuracy.

Noticed by:
Included it: Meta, DeepSeek. Did not include: Claude, ChatGPT, Gemini, Nova, Wenxin, Qwen.
Source ↗

04Questions people ask

AI models vs AI agents?

A model is the trained network; an agent is that model plus a harness, tools and memory working in a loop. The harness alone can move a score from 45.9% to 55.4%.

Source ↗
What is the best AI model for agents?

Among open weights, DeepSeek V4.1 Flash leads the Vals Index at 61.26% for $0.66 a test. For legal agents, Muse Spark Open tops Harvey's benchmark at 25.00%.

Source ↗
What is the best AI model for Hermes Agent?

Nous Research names no official default, and guides disagree (Kimi K3, GLM-5.2, MiniMax M3, a small Qwen for local use). Test in your own Hermes harness.

Source ↗
Why do vendor agent scores and independent scores differ?

Vendors report their best harness and settings, and often an older test version. DeepSeek's vendor Terminal-Bench 4.0 figure is 42.3%, while Vals measured 31.8%.

Source ↗

05How we counted

In plain words, no formulas.

1The question

Top 10 Open-Weight Models for AI Agents: Who Finishes the Job, Not Just Who Wins the Press Release

2Why this topic

A person building agents sees which open models hold up outside the vendor's own lab, and where a launch headline and an independent test disagree.

3Where it applies

Global open-weight models used as agents for coding, security and legal work

4What we looked at

  • Agent task success on independent benchmarks
  • Gap between the vendor-reported agent score and the nearest independent score
  • Cost of one finished task
  • Model size and hardware needed to run it
  • Vals AI (Vals Index, Harvey Legal Agent Benchmark, Terminal-Bench 4.0, Finance Agent) — Independent accuracy on professional agent tasks, with cost per test
  • Artificial Analysis (FrontierCode 2.0, Coding Agent Index, Cyber Index) — Independent completion of coding and cyber agent tasks, with cost per test attempt
  • Vendor-reported agent scores — Launch-day self-reported results (CyberGym, Cybench, DeepSWE, Terminal-Bench 2.1), measured with each vendor's own harness
  • Model size and hardware — Total and active parameters, and the GPUs needed to run the model

5Who ranked

The leading AIs, each building its own list of ten independently and naming a source for every place: Claude, ChatGPT, Meta, Gemini, Nova, Wenxin, DeepSeek, Qwen.

6How the places were set

The same entry written differently by different AIs was counted as one. The more AIs put an entry on their list, and the higher they put it, the higher its place. "AI agreement" shows how many of the AIs had it on their own list. Sources are only those the AIs found themselves; nothing was added by hand.

06Each AI's own list

🇺🇸 Claude10 places · 3 notes

The same question, answered by Claude alone. The final list above combines all the AIs.

  1. 1
    GLM-5.3 (Z.ai, 2026)

    Z.ai published its test method (Claude Code 2.1.207, pass@1 over 1,507 CyberGym tasks) so anyone can contest its 84.5, but nobody independent has yet.

  2. 2
    Mistral Large 4 (2026)

    Its cyber claim has the smallest visible gap: Mistral says ~82% on CyberGym-E2E, an independent index measured 81.7%.

    Source ↗
  3. 3
    DeepSeek V4.1 Flash (2026)

    Mistral's chart left it out, and Reflection's table puts it above Large 4 on DeepSWE (74.2% vs 61.7%).

    Source ↗
  4. 4
    Kimi K3 (Moonshot AI, 2026)

    Its 88.3 on Terminal-Bench 2.1 is Moonshot's own, while Meta's table lists 85.0, and the weights take 1.4 TB.

    Source ↗
  5. 5
    Muse Spark Open (Meta, 2026)

    Meta says it beats Kimi K3 on tool use but trails on coding: 76.4 vs 85.0 on Terminal-Bench 2.1.

    Source ↗
  6. 6
    Qwen3.8-2.4T-A95B (Alibaba, 2026)

    The 92.6 GPQA claim is for the Max version, not the downloadable checkpoint.

  7. 7
    MiMo-V2.6-Pro (Xiaomi, 2026)

    A Vals write-up headlines it at about 100 times cheaper than Claude Opus 5.5.

  8. 8
    Nemotron 3.5 Bolt 74B-A9B (NVIDIA, 2026)

    The lightest of the group: 74B with 9B active, fitting in 24 GB at 2-bit.

  9. 9
    Qwen3.8 27B (Alibaba, 2026)

    A laptop-class agent brain at 34 on the old index, but only 20 on v5.

    Source ↗
  10. 10
    Kolibri-1 (Aleph Alpha, 2026)

    It runs on two 80 GB A100s but scores only 27.7 on Terminal-Bench 2.1, its own figure.

What Claude noticed

Mistral's cyber claim is the one place where a launch-day number almost matches an independent one (~82% claimed, 81.7% measured). But the argument rests partly on closed models refusing the tasks, and that gap may shrink once weights are public.

Harness choice moves agent scores as much as model choice. One model scored 45.9% on SWE-Bench Pro under a standard scaffold and 55.4% under its vendor's own agent, and Scale AI saw a 5.2-point spread across three agents.

The vendor Terminal-Bench 2.1 scores for open models are 76 to 88, while the best open-harness record on Terminal-Bench 2 is 51.9%. The two use different versions and harnesses, so the gap is a warning, not a measurement.

🇺🇸 ChatGPT10 places · 2 notes

The same question, answered by ChatGPT alone. The final list above combines all the AIs.

  1. 1
    DeepSeek V4.1 Flash 1017 (2026)

    Its 28% independent FrontierCode completion leads released open weights, and one measured provider charges $0.37 per composite intelligence-test attempt.

    Source ↗
  2. 2
    GLM-5.3 (2026, Z.ai)

    Matches DeepSeek's intelligence index at 32 but costs $2.53 per intelligence-test attempt at one independently measured provider.

    Source ↗
  3. 3
    MiMo-V2.6-Pro (2026, Xiaomi)

    Combines 57.81% Vals professional accuracy with a reported $0.24 cost per intelligence-test attempt at GMI.

    Source ↗
  4. 4
    Mistral Large 4 OSS (2026, France)

    Its 81.56% independent Vibe Code result contrasts with just 26.26% on Terminal-Bench 4.0: task selection changes the story dramatically.

  5. 5
    MiMo-V2.6-Flash (2026, Xiaomi)

    At $0.11 per intelligence-test attempt on GMI, it is a low-cost candidate for repeated agent retries.

    Source ↗
  6. 6
    Ling 3.1 Flash 1019 (2026)

    Its 29-point intelligence result and 25 billion active parameters put it among the stronger efficient agent candidates.

    Source ↗
  7. 7
    Step 5 Preview (2026, StepFun)

    Its 29-point index matches Ling, but its approximately $0.50 blended token price weakens the economics of repeated attempts.

    Source ↗
  8. 8
    Kimi K3 (2026, Moonshot AI)

    Its strong human-preference showing makes it worth testing as a planning or review agent, but that is not proof of completed tool workflows.

  9. 9
    MiniMax-M3.1-Flash (2026)

    A 29-point intelligence score and a $0.42 measured test-attempt cost make it interesting, although deployment rights remain unclear.

    Source ↗
  10. 10
    Mistral Large 3 (2025–2026)

    Apache 2.0 offers an operational advantage when a business needs to host customer agents without Large 4's Model-as-a-Service exception.

    Source ↗

What ChatGPT noticed

On FrontierCode 2.0, the best released open-weight model reaches 28%, compared with 46% for Claude Opus 5.5. The 18-point gap is a direct warning against equating general intelligence with autonomous coding reliability.

A $0.11 test attempt is not automatically cheaper than a $0.37 attempt if the cheaper model fails much more often. Finished-task economics require measuring retries and human repair.

🇺🇸 Meta10 places · 4 notes

The same question, answered by Meta alone. The final list above combines all the AIs.

  1. 1
    DeepSeek V4.1 Flash 1017 (2026, China)

    Vals Index open leader 61.26% at $0.66 per test vs Sonnet 5.5 at $18.70 – 28x cheaper, 90.6 Terminal-Bench 2.1 vendor but 31.8 independent

    Source ↗
  2. 2
    Xiaomi MiMo-V2.6-Pro (2026, China)

    89.9 Terminal-Bench 2.1 just above Claude Opus 5 at 89.1, 71.9 DeepSWE v1.1, 94% CyberGym, $0.13 per Intelligence task

    Source ↗
  3. 3
    Z.ai GLM-5.3 (2026, China)

    84.5% CyberGym white-box – technically ahead of Claude Mythos 5 at 83.8%, DeepSWE 66.9% vs 46.2 prior, 40% Terminal-Bench 4.0

    Source ↗
  4. 4
    Z.ai GLM-5.3 Flash (2026, China)

    MIT, $0.15/$0.50, 84.3 Terminal-Bench 2.1 vendor, best cost to run open agent on single-host

    Source ↗
  5. 5
    Mistral Large 4 Le Chonk (2026, France)

    Vendor 93% Cybench, 82% vulnerability patching, 61.7% DeepSWE v1.1 – independent 41% Terminal-Bench 4.0 and 48% Coding Agent Index using 47% fewer tokens

    Source ↗
  6. 6
    Moonshot Kimi K3 (2026, China)

    Top open on Vals Index at 57%, 1488 Arena, K3.1 Pro scores 67% Terminal-Bench 4.0 in v5 – best open terminal score

    Source ↗
  7. 7
    DeepSeek V4 Pro 0813 (2026, China)

    MIT, zero refusals on 840 adversarial security tasks, 76% CyberGym-E2E vs 79% for Mistral Large 4, half cost per success

    Source ↗
  8. 8
    Alibaba Qwen3.8-2.4T-A95B Max (2026, China)

    2.4T total 95B active, custom $50M MaaS gate, 68.89% IOI, Apache 27B sibling is clean license fallback

    Source ↗
  9. 9
    MiniMax M3.1 Flash (2026, China)

    428B total 23B active MoE, $0.075/$0.30 cheapest frontier agent, 43 on Intelligence Index, weights on Hugging Face Oct 23

    Source ↗
  10. 10
    Google Gemma 4 31B / Nvidia Nemotron 3.5 Bolt 74B (2026, US)

    Best US hardware fit – Gemma 1452 Arena on 31B, Nemotron 74B total 9B active fits 24GB GPU, Apache 2.0 + NVIDIA open license

    Source ↗

What Meta noticed

Trust gap is 10+ points: DeepSeek V4.1 Flash vendor 42.3% Terminal-Bench 4.0 vs Vals independent 31.8%, Mistral Large 4 vendor 82% vulnerability patching vs 41% Terminal-Bench 4.0 independent – launch claims sit 20 points above independent reruns

Cost per finished task: DeepSeek V4.1 Flash 1017 at $0.66 per Vals test vs Claude Sonnet 5.5 at $18.70 and Opus 5.5 at $28.12 – 28x and 42x cheaper for 9 points lower accuracy

CyberGym is saturated – GLM-5.3 at 84.5% vs Claude Mythos 5 at 83.8% and GPT-5.6 Sol at 83.6% within 7.3 points – capability gap not visible in headline percentage

Terminal-Bench is the separator – best open Kimi K3.1 Pro at 67% vs best released DeepSeek V4.1 Flash at 28% FrontierCode and 13% Terminal-Bench-Science, most open models under 10%

🇺🇸 Gemini10 places

The same question, answered by Gemini alone. The final list above combines all the AIs.

  1. 1
    Qwen 2.5 Coder 32B Instruct (Alibaba, China)

    Has the lowest trust gap in the industry (under 4 percentage points), completing complex coding agent runs for pennies on consumer hardware.

    Source ↗
  2. 2
    DeepSeek-V3 (China, 2024–2026)

    Balances massive 671B MoE tool-calling capability with an aggressive sub-5% trust gap and negligible API invocation pricing.

    Source ↗
  3. 3
    Llama 3.3 70B Instruct (Meta, USA)

    Finishes 92% of function-calling sequences without syntax degradation, exhibiting a tight 5.1% variance from Meta's whitepaper claims.

  4. 4
    Nous Hermes 3 - 70B (Nous Research, USA)

    Engineered purely for autonomous agent steering and JSON function execution, maintaining near-zero prompt-drift across 50-step runs.

    Source ↗
  5. 5
    Mistral Large 4 (Mistral AI, France, 2026)

    Claims a breathtaking 82% on CyberGym-E2E and 61.7% on DeepSWE v1.1, but drops to 15.83% on Harvey's independent Legal Agent Benchmark.

    Source ↗
  6. 6
    Llama 3.1 405B Instruct (Meta, USA)

    Highest raw reasoning headroom among open weights, but requires an 8-GPU data center node to run a single agent thread.

  7. 7
    DeepSeek-Coder-V2.5 (China)

    A reliable automated bug-hunter whose 16B active MoE design slashes completed coding patch costs down to $0.05.

    Source ↗
  8. 8
    Qwen 2.5 72B Instruct (Alibaba, China)

    A dependable general agent that powers multi-agent orchestrators with 128k context stability and negligible parsing failures.

    Source ↗
  9. 9
    Devstral 24B / Mistral NeMo (Mistral AI, France)

    Mistral's nimble local agent runner that executes IDE terminal commands reliably without burning enterprise cloud credits.

    Source ↗
  10. 10
    GLM-4 9B Chat (Zhipu AI, China)

    The entry ticket for mobile and edge agent setups, capable of parsing web-browsing actions within 16GB RAM limits.

    Source ↗
🇨🇳 Wenxin10 places · 1 notes

The same question, answered by Wenxin alone. The final list above combines all the AIs.

  1. 1
    Meta Llama 4 Maverick 405B (2026)

    405B agent beast — but you need 8×H100s and 12k hardware to let it finish a job

  2. 2
    DeepSeek-V3.2 671B (2025/2026)

    MoE trick cuts active params to ~37B — the agent that thinks big, spends little

  3. 3
    Mistral Large 4 123B (Mistral AI, France, 2026)

    123B agent scores look great on paper — independent tests show a 20-point gap

    Source ↗
  4. 4
    Qwen3 235B-A22B MoE (Alibaba, 2026)

    22B active at inference — the agent you can actually afford to run 24/7

    Source ↗
  5. 5
    Falcon 3 180B (TII, UAE, 2026)

    180B generalist — decent agent but not the specialist you need for code or law

  6. 6
    NVIDIA Nemotron 4 340B (2025/2026)

    Nvidia's own agent model — built to showcase their chips, not your budget

  7. 7
    Microsoft Phi-4 14B (2025/2026)

    14B on a laptop — the agent that runs while your coffee brews

  8. 8
    Cohere Command R+ 104B (2025/2026)

    Built for enterprise RAG agents — but 'open' only if you don't want to sell it

    Source ↗
  9. 9
    Stability AI Stable Code 3 15B (2026)

    15B code agent — great at GitHub tickets, can't do your taxes

  10. 10
    Google Gemma 3 27B (2026)

    27B generalist agent — good enough for 80% of tasks at 1/10th the cost

    Source ↗

What Wenxin noticed

DeepSeek-V3.2 costs under 1 per million tokens self-hosted, making it the cheapest top-5 agent model — and it's a MoE, so only ~37B parameters are active per token

🇨🇳 DeepSeek10 places · 1 notes

The same question, answered by DeepSeek alone. The final list above combines all the AIs.

  1. 1
    DeepSeek V4.1 Flash 1017 (2026)

    A 61.26% independent Vals Index score at $0.66 per test — but its vendor-reported 74.2 DeepSWE collapses to 19.7% on Vals’ independent Terminal-Bench 4.0, one of the widest trust gaps in the field.

  2. 2
    GLM-5.3 (2026)

    Vals’ coding category gives it 64.3 — the best open-weight coding-agent score — and its independent numbers hold up better than most, but its license dispute matters for anyone hosting agents for customers.

    Source ↗
  3. 3
    Mistral Large 4 (Le Chonk, 2026)

    Mistral’s launch claims 82% CyberGym-E2E, 93% Cybench and 61.7% DeepSWE — but independent Cyber Index testing puts it around 51%, a ~31-point trust gap that is the defining number for security-agent buyers.

    Source ↗
  4. 4
    Kimi K3 (Moonshot AI, 2026)

    A 57.8% independent Vals score and a 1,487 LMArena open-weight Elo, but the $20M revenue trigger and 100M-MAU cap mean agent platforms must budget for a commercial deal, not just GPUs.

    Source ↗
  5. 5
    MiMo-V2.6-Pro (Xiaomi, 2026)

    A 59.5% independent Vals score with full MIT licensing and $0.435/$0.87 per million tokens — the best cost-per-finished-task option among genuinely permissive models.

    Source ↗
  6. 6
    Qwen 3.8 Max (Alibaba, 2026)

    Strong coding-agent numbers (67.4 on Terminal-Bench 2.1 in vendor tests) but a $50M revenue gate for competing MaaS and coding-assistant businesses — the agent platform that hosts it for customers must plan a commercial deal.

    Source ↗
  7. 7
    Muse Spark Open (Meta, 2026)

    Ranks first on Vals AI’s Harvey Legal Agent Benchmark at 25.00%, but its $20M MaaS-resale trigger means legal-tech platforms need a separate deal before hosting it for clients.

  8. 8
    Step 5 Preview (StepFun, 2026)

    A 29 on AA v5 with 1M-token context and a 600B MoE design, but the StepFun Model License 1.0 carries OEM and MaaS conditions that can block agent deployment even though the weights download.

  9. 9
    Nemotron 3 Ultra (NVIDIA, 2026)

    Only a 23 on AA v5, but OpenMDW-1.1 opens weights, training data and recipes — the most transparent long-running-agent model, at $0.60/$2.60 per million tokens.

  10. 10
    Beam 1.0 (Reflection, 2026)

    A 30 on AA v5 and 48.5% on AutomationBench-AA — strong for agentic workflows — but only 3.0% on Terminal-Bench 4.0, so it finishes knowledge-work agents, not terminal agents.

What DeepSeek noticed

Open-weight agents can match closed models at less than a tenth of the cost if they are given enough turns — a finding that changes the buy-vs-build math for security and legal agent teams.

🇨🇳 Qwen10 places · 2 notes

The same question, answered by Qwen alone. The final list above combines all the AIs.

  1. 1
    Kimi K3.1 Pro

    Moonshot's agent finishes 94.7% of real-world hacking tasks, leaving a massive gap between its independent score and everyone else's press release.

    Source ↗
  2. 2
    DeepSeek V4.1

    The MIT-licensed workhorse completes complex multi-file coding agents at a fraction of the cost of US frontier models.

  3. 3
    Mistral Large 4

    Mistral claims 82% on CyberGym and 61.7% on DeepSWE, but independent testers measure 81.7% and 60%, showing a rare 'honest' trust gap.

    Source ↗
  4. 4
    GLM-5.1

    Post-trained specifically for Harvey's Legal Agent Benchmark, it drafts and reviews contracts faster than any general-purpose model.

  5. 5
    Qwen 3.8 Max

    Alibaba's agent orchestrator dominates Asian legal and financial workflows, though its $50M revenue license cap slows down global SaaS adoption.

  6. 6
    Llama 3.3 405B

    Meta's massive context window lets it hold entire codebases in memory for long-horizon refactoring agents.

  7. 7
    Yi-Lightning

    01.AI's lightweight agent runs on cheap cloud GPUs, making it the default for startups building automated customer support loops.

    Source ↗
  8. 8
    MiMo

    A specialized reasoning agent that beats general models on math-heavy coding tasks without needing a 100B parameter footprint.

    Source ↗
  9. 9
    Phi-4

    Microsoft's small agent excels at local, on-device security sweeps where sending code to the cloud is a compliance violation.

    Source ↗
  10. 10
    InternLM 3

    Shanghai AI Lab's agent framework includes built-in tool-use tracing, making it the favorite for audited financial orchestration.

    Source ↗

What Qwen noticed

The 'Trust Gap' in AI agents is narrowing: Mistral Large 4's vendor claims (82% on CyberGym) were nearly identical to independent tests (81.7%), a rare moment of transparency in a field plagued by cherry-picked benchmarks.

Hardware efficiency is the new moat for agents: Phi-4 and Yi-Lightning allow startups to run 24/7 autonomous security sweeps on a single RTX 4090, bypassing cloud API rate limits entirely.