Back to the trend of the day: Meta says its cheap AI matches Anthropic's for a third the cost
Poly Rating · from the trend of the day

Best Model for AI Agents 2026: Office Work vs Code

Muse Ember finishes 68.1% of office-workflow objectives to Claude Haiku 5.5's 35.4%, yet Haiku's patches pass maintainers' tests more often, 34.9% to 27.2%.

  • No. 1: Meta Muse Ember
  • 10 places

01Final list

The leading AIs each built their own list of ten independently. Here is one result that is easy to check: every place has a reason, a source, and the list of AIs that included it.

Order: from most to least AI agreement.

A bright badge means this AI put the entry on its list. A dim badge means it did not.

  1. 1

    Meta Muse Ember

    Give it the office steps: it finishes 68.1% of workflow objectives against Haiku's 35.4%. Keep it off the final merge, where it scores 27.2% to Haiku's 34.9%.

    Included it: Claude, ChatGPT, Meta, Grok, Z.ai, Kimi, DeepSeek, Qwen. Did not include: Gemini, Wenxin.

    In their lists: Claude: 1 · ChatGPT: 1 · Meta: 5 · Grok: 1 · Z.ai: 3 · Kimi: 2 · DeepSeek: 1 · Qwen: 2

    What each AI noted 8
    • Claudeplace 1

      Wins office work at 37.8% on AutomationBench-AA (Haiku 13.9%), but loses code: 27.2% vs 34.9% on FrontierCode 2.0.

      Source ↗
    • ChatGPTplace 1

      The office specialist: 68.1% workflow-objective completion, but only 27.2% on the mergeability-oriented coding evaluation.

      Source ↗
    • Metaplace 5

      Terminal-first: 40.9% Terminal-Bench in Meta's table, 27.2% FrontierCode-AA - office score not in the AA AutomationBench 24-model list found.

      Source ↗
    • Grokplace 1

      Give it the office steps: 68.1% of workflow objectives, then keep it off the final merge.

      Source ↗
    • Z.aiplace 3

      The office-code split in a single model: 68.1% on AA's workflow run (Meta's own stricter table says 37.8), a terminal win over Haiku 40.9% to 32.8% - and a loss to Haiku on merge-ready code 27.2% to 34.9%.

      Source ↗
    • Kimiplace 2

      Finishes 37.8% of office workflows — 2.7x Haiku's rate — but the maintainer's-merge test flips it: 27.2% vs Haiku's 34.9%.

      Source ↗
    • DeepSeekplace 1

      The only cheap model that is simultaneously top-tier on business workflows (37.8% strict) and competitive on terminal work (40.9%), though only 27.2% of its coding patches would be merged.

      Source ↗
    • Qwenplace 2

      Built specifically for the repetitive API calls and routing agents do all day.

      Source ↗
    8 AIs put it here
  2. 2

    Claude Haiku 5.5

    The cheap model to let touch the code: 34.9% on AA's FrontierCode run, though Cognition's own board gives 27.7%. On office work it finishes only 35.4%.

    Included it: Claude, ChatGPT, Meta, Grok, Z.ai, Kimi, DeepSeek, Qwen. Did not include: Gemini, Wenxin.

    In their lists: Claude: 2 · ChatGPT: 2 · Meta: 4 · Grok: 2 · Z.ai: 5 · Kimi: 1 · DeepSeek: 2 · Qwen: 4

    What each AI noted 8
    • Claudeplace 2

      The code pick: best FrontierCode 2.0 score in the small group (34.9%), yet it finishes only 13.9% of office workflows.

      Source ↗
    • ChatGPTplace 2

      It loses the office-workflow contest to Ember but beats it by 7.7 points on the reported FrontierCode comparison.

      Source ↗
    • Metaplace 4

      The cheap code leader: 27.7% FrontierCode 2.0 and 36% terminal, but only 35.4% on office workflows.

      Source ↗
    • Grokplace 2

      The cheap model that should touch the code: 34.9% merge-ready, 13.9% strict on the office test.

      Source ↗
    • Z.aiplace 5

      The one to let touch the code: 34.9% of patches clear maintainers' tests on FrontierCode 2.0, about ten points over every cheap rival - while its office score (35.4%) is the weakest here.

      Source ↗
    • Kimiplace 1

      The only small model that wins the code column — FrontierCode 2.0 at 34.9% beats Muse Ember by 7.7 points — while taking 39.2% on Terminal-Bench.

      Source ↗
    • DeepSeekplace 2

      The merge-ready coder of the cheap class: 34.9% on FrontierCode 2.0, but only 13.9% strict on office workflows — it will fix the code and ignore the spreadsheet.

      Source ↗
    • Qwenplace 4

      Anthropic's safety guardrails keep it from violating SaaS workflow rules.

    8 AIs put it here
  3. 3

    DeepSeek V4.1 Flash

    Strong on office work at 66.6%, but only 19.3% on Cognition's code board. Its terminal score falls from a self-reported 42.3% to 31.82% in Vals' run.

    Included it: Claude, ChatGPT, Meta, Grok, Z.ai, Kimi, DeepSeek. Did not include: Gemini, Wenxin, Qwen.

    In their lists: Claude: 4 · ChatGPT: 5 · Meta: 8 · Grok: 5 · Z.ai: 4 · Kimi: 4 · DeepSeek: 3

    What each AI noted 7
    • Claudeplace 4

      Best long-session coder in Bito's group (71.9%) at $2.17 per 20-turn run; no AutomationBench-AA figure found.

      Source ↗
    • ChatGPTplace 5

      A 33-point intelligence score and open weights are appealing, but FrontierCode drops to 19.3% on harder merge-ready work.

      Source ↗
    • Metaplace 8

      Terminal-strong for its price at 31.82% for $0.54, code-weak at 19.3% - a shell worker, not a merger.

    • Grokplace 5

      On the Hermes board it scores 36.91 for $0.26 a task, next to Haiku's 37.00 at $0.49.

      Source ↗
    • Z.aiplace 4

      66.6% on office workflows and open MIT weights to self-host - but its terminal story flips with the harness: 42.3% self-reported versus 19.7% in Vals' independent run.

      Source ↗
    • Kimiplace 4

      Cheapest serious code option: 19.3% on FrontierCode 2.0 at $0.38 per rollout, with Terminal-Bench at 42.3% self-reported / 33.8% independently.

      Source ↗
    • DeepSeekplace 3

      The terminal specialist: 42.3% on Terminal-Bench 4.0 and 60.5% on AutomationBench v1.0.6, but only 28% on FrontierCode — strong at running commands, weaker at shipping mergeable patches.

      Source ↗
    7 AIs put it here
  4. 4

    GPT-6 Luna

    Middling on office work and weak at the terminal: 53.2% of office objectives (20.2% strict), 25.2% on code, and only 16.4% of terminal tasks resolved.

    Included it: Claude, ChatGPT, Meta, Grok, Z.ai, Kimi, DeepSeek. Did not include: Gemini, Wenxin, Qwen.

    In their lists: Claude: 6 · ChatGPT: 6 · Meta: 6 · Grok: 9 · Z.ai: 8 · Kimi: 7 · DeepSeek: 6

    What each AI noted 7
    • Claudeplace 6

      Middle on office work (20.2%) and last of the three launch rivals on code (25.2%) and terminal tasks (16.4%).

      Source ↗
    • ChatGPTplace 6

      A low-cost helper with 31 on the Coding Agent Index, but only 16.4% official terminal-task resolution.

      Source ↗
    • Metaplace 6

      Balanced-cheap: 53.2% office and 21.9% code - better office than Haiku, weaker code than Haiku.

      Source ↗
    • Grokplace 9

      Loses the office strict test at 20.2% and the merge test at 25.2%, on a sticker that matches Ember.

      Source ↗
    • Z.aiplace 8

      Its office score nearly triples depending on who scores it - 20.2% in Meta's single-attempt table versus 53.2% on AA's standard run - and mergeability sits at 23.8% on Cognition's board.

      Source ↗
    • Kimiplace 7

      The office-over-code split in one model: strong on business workflows (AutomationBench-AA 53% in one AA run, 20.2% in Meta's stricter chart) but 16.4% on Terminal-Bench.

      Source ↗
    • DeepSeekplace 6

      A low per-task bill ($0.15) hides weak office and terminal results: 20.2% AutomationBench strict, 16.4% Terminal-Bench, and 25.2% FrontierCode.

      Source ↗
    7 AIs put it here
  5. 5

    MiniMax M3.1 Flash

    The best office score among cheap models, 72.5%. Its terminal score drops from a self-reported 38.1% to 25.25% in Vals' run: use it for apps, not shells.

    Included it: Claude, Meta, Grok, Z.ai, Kimi, DeepSeek. Did not include: ChatGPT, Gemini, Wenxin, Qwen.

    In their lists: Claude: 5 · Meta: 2 · Grok: 7 · Z.ai: 2 · Kimi: 5 · DeepSeek: 5

    What each AI noted 6
    • Claudeplace 5

      Reaches 75.0% on Bito's 20-turn sessions, nearly level with Haiku 5.5, but needs 3 to 5 hours per run.

      Source ↗
    • Metaplace 2

      72.5% of office objectives finished - the best office score among cheap models - next to a 25% terminal score.

      Source ↗
    • Grokplace 7

      The launch terminal number, 38.1%, and the shared-harness number, 25.25%, are not the same test.

      Source ↗
    • Z.aiplace 2

      72.5% of office-workflow objectives - second only to Gemini 4 Carbon among 249 models - while Vals billed it just $0.21 per terminal test, the cheapest finished-task bill in the class.

      Source ↗
    • Kimiplace 5

      38.1% on Terminal-Bench 4.0 — 22 points above Luna at the same ~$0.60 output price, with no office or merge-code numbers published yet.

      Source ↗
    • DeepSeekplace 5

      The office-workflow champion of the cheap set at 72.5% AutomationBench-AA, but only 21% FrontierCode and 26% Terminal-Bench — route it the spreadsheets, not the pull request.

      Source ↗
    6 AIs put it here
  6. 6

    GLM-5.3-Flash

    An open-weights model for the office queue at 60.4%, with mergeable code down at 14.2% on Cognition's board and 17.6% in AA's run.

    Included it: ChatGPT, Grok, Kimi, DeepSeek. Did not include: Claude, Meta, Gemini, Z.ai, Wenxin, Qwen.

    In their lists: ChatGPT: 4 · Grok: 3 · Kimi: 6 · DeepSeek: 7

    What each AI noted 4
    • ChatGPTplace 4

      35.8% terminal resolution for a Flash model, but only 14.2% on the maintainer-quality coding benchmark.

      Source ↗
    • Grokplace 3

      An open model for the office queue, at 60.4%, with merge-ready code down at 17.6%.

      Source ↗
    • Kimiplace 6

      35.8% on Terminal-Bench 4.0 for $0.36 a task — the price-performance point where cheap terminal work starts being reliable.

      Source ↗
    • DeepSeekplace 7

      A 60% AutomationBench score makes it a credible office step-runner, but 15% FrontierCode and 33% Terminal-Bench keep it away from repository work.

      Source ↗
    4 AIs put it here
  7. 7

    Gemini 3.8 Flash

    Finishes 59.9% of office objectives but only about 19% of terminal tasks, and Vals billed $8.65 per terminal test, the highest bill in the budget group.

    Included it: ChatGPT, Meta, Z.ai, DeepSeek. Did not include: Claude, Grok, Gemini, Kimi, Wenxin, Qwen.

    In their lists: ChatGPT: 7 · Meta: 3 · Z.ai: 7 · DeepSeek: 9

    What each AI noted 4
    • ChatGPTplace 7

      A multimodal million-token assistant, yet only 19.1% of official terminal tasks were resolved in its evaluated harness.

      Source ↗
    • Metaplace 3

      59.9% office, 19.7% terminal - the same split as MiniMax, at a far higher measured task cost.

      Source ↗
    • Z.aiplace 7

      59.9% on office workflows with intelligence fans compare to Opus-tier - yet the same model cost Vals $8.65 per terminal test, the biggest finished-task bill in the budget class.

      Source ↗
    • DeepSeekplace 9

      A 60% AutomationBench score at high effort, but only 19% FrontierCode and 20% Terminal-Bench — a reminder that office competence does not transfer to merge-ready code.

      Source ↗
    4 AIs put it here
  8. 8

    Thinking Machines Inkling 2 Small

    Reported to jump to 46% on office work, from a single source, with code at 22.6% to 25%. It resolves only 12.63% of terminal tasks.

    Included it: Claude, ChatGPT. Did not include: Meta, Grok, Gemini, Z.ai, Kimi, DeepSeek, Wenxin, Qwen.

    In their lists: Claude: 3 · ChatGPT: 8

    What each AI noted 2
    • Claudeplace 3

      Reported to jump from 5% to 46% on AutomationBench-AA over its predecessor, while Terminal-Bench 4.0 stays at about 13%.

      Source ↗
    • ChatGPTplace 8

      Its 22.6% FrontierCode score is respectable, but a $1.18 index task and slow completion weaken the agent economics.

      Source ↗
    2 AIs put it here
  9. 9

    Qwen3.8 Flash

    Its office score of 55.9% sits 30 points above its 25.3% terminal score on the same Artificial Analysis page. Only one source reports these figures.

    Included it: Grok. Did not include: Claude, ChatGPT, Meta, Gemini, Z.ai, Kimi, DeepSeek, Wenxin, Qwen.

    In their lists: Grok: 4

    What each AI noted 1
    • Grokplace 4

      Office work at 55.9% sits 30 points above its terminal score on the same card.

      Source ↗
    1 AIs put it here
  10. 10

    Qwen3.8-27B

    A specialist in on-screen desktop work, not the terminal: 84.3% on OSWorld-Verified but only 5.1% on Terminal-Bench 4.0, with 48.2% on office work. Only one source reports these figures.

    Included it: Z.ai. Did not include: Claude, ChatGPT, Meta, Grok, Gemini, Kimi, DeepSeek, Wenxin, Qwen.

    In their lists: Z.ai: 9

    What each AI noted 1
    • Z.aiplace 9

      The GUI specialist of the small tier: 84.3% on OSWorld-Verified (fourth overall) against 5.1% on Terminal-Bench 4.0 - desktop apps yes, containerized terminals no.

      Source ↗
    1 AIs put it here

More about the top three ↓

02More about the top three

The ranking is above. Here is more about the three places at the top: what they are, why they are here, and what is worth knowing.

The three are compared on different tests: Ember's 68.1% and DeepSeek's 66.6% come from office-workflow runs, and Haiku's 35.4% comes from the same office measure. The code and terminal scores come from other benchmarks, and some are self-reported or from inde

1Meta Muse Ember

No. 1 — Meta Muse Ember. On the lists of 8 of 10 AIs; median place there: 1.5. The AIs put it first because it finishes far more office workflow steps than Haiku, even though it loses on merge-ready code.

Did you know

  • Meta Muse Ember is the smallest model in Meta's Muse family. It is meant to run alongside a bigger planner, Muse Wonder. Meta's line: "Muse Wonder plans, Muse Ember handles the volume." Its weights are kept private.

    Source ↗ ✓ confirmed by several AIs
What it is
Meta's smallest, cheapest Muse model, built for small, repeated agent jobs.
Its edge
It finishes 68.1% of office workflow objectives, well above Haiku's 35.4%.
Who it suits
Suits office-style agent work; not for code that must be merged.
  • Office workflow objectives completed68.1% of office objectives
Why it is here
  • On Meta's long desktop-workflow test (OSWorld 2.1), Ember scores 69.5% partial credit. Claude Haiku 5.5 scores 68.9%. That is nearly a tie, so Ember's office-work lead is small on this measure.

  • On fully finished tasks, Ember completes 33.8% against Haiku's 27.8%, per The New Stack. Its edge shows up when you count only tasks done start to finish.

Did you know
  • Ember costs $0.10 per million input tokens and $0.50 per million output tokens. It still has a 1M-token context window and takes image inputs. It is the fourth Muse model, after Spark, Glimmer and Wonder.

Keep in mind
  • Ember launched on November 9, 2026 as a smaller, cheaper model, so coverage so far leans on Meta's own benchmarks. I found no independent user reviews yet. Treat the early scores as the maker's claims.

Where to go further
  • Open the blog and try Muse Ember from the 'Try Muse' button. Look first at the 'Intelligence vs. Cost Per Task' chart right after the intro. These numbers are Meta's own, so check them against independent tests.

Compare the top three ↓

2Claude Haiku 5.5

No. 2 — Claude Haiku 5.5. On the lists of 8 of 10 AIs; median place there: 2. The AIs rank it second as the cheap model to trust with code, since more of its patches pass maintainers' tests, despite weaker office scores.

What it is
Anthropic's small, low-cost model in the Claude 5.5 family.
Its edge
Its code patches pass maintainers' tests more often: 34.9% against Ember's 27.2%.
Who it suits
Suits code fixes and patches; not the best pick for office workflows.
  • Office workflow objectives completed35.4% of office objectives

Wikidata

Why it is here
  • Meta's results page lists Claude Haiku 5.5 at 68.9% partial credit on the OSWorld 2.1 office-style workflows. Ember is at 69.5%, so Haiku stays close behind on desktop tasks.

  • The same test counts full completion separately. Haiku finishes 27.8% of tasks completely, compared with 33.8% for Ember. Part of its place comes from strength elsewhere, such as code.

Kimi put it first. What they saw in it
  • Kimi: The only small model that wins the code column — FrontierCode 2.0 at 34.9% beats Muse Ember by 7.7 points — while taking 39.2% on Terminal-Bench.

Compare the top three ↓

3DeepSeek V4.1 Flash

No. 3 — DeepSeek V4.1 Flash. On the lists of 7 of 10 AIs; median place there: 4. The AIs place it third because its office score is strong, but its code score is weak and its terminal results vary by test.

What it is
DeepSeek's cheap Flash model with open weights, now also serving its Pro requests.
Its edge
Close to Ember on office work at 66.6%, and cheaper per task than Haiku.
Who it suits
Suits self-hosting and terminal tasks; not for merge-ready code.
  • Office workflow objectives completed66.6% on office workflows
Why it is here
  • On Vals' independent Terminal-Bench 4.0 run, the DeepSeek V4.1 Flash 1017 version scores 31.82%. GPT-6.1 Sol scores 55.05% on the same page, so the gap to the leader is large.

History
  • On 14 September 2026, DeepSeek began sending every request for its top V4-Pro model to the cheaper V4.1-Flash instead, passing the savings on to customers. Flash had launched four days earlier, on 10 September.

    Source ↗ ✓ confirmed by several AIs
Keep in mind
  • DeepSeek says all requests to its Pro model are now routed to V4.1 Flash and billed at Flash's price. Commenters on Hacker News call silent model swaps concerning, and one user reports Flash failing in coding tools with loops and invalid tool calls.

Where to go further
  • Open the model card on Hugging Face. Start with the 'Performance across agent scaffolds' section, which covers the terminal-task tests. The page also lists 552B parameters while Hub metadata says 763B, so read the numbers carefully.

Compare the top three ↓

The top three side by side

Meta Muse EmberClaude Haiku 5.5DeepSeek V4.1 Flash
In the AIs' lists8 / 108 / 107 / 10
Office workflow objectives completed68.1% of office objectives35.4% of office objectives66.6% on office workflows
Put first byClaude, ChatGPT, Grok, DeepSeekKimi—

Places with the same number of AIs are ordered by their average place in the AIs' own lists.

03Worth knowing

What the AIs noticed while building the list.

The office leader is not the code leader. Ember beats Haiku 5.5 on finished workflows (68.1% vs 35.4%) and loses on merge-ready code (27.2% vs 34.9%). Both results are Artificial Analysis runs.

Noticed by:
Included it: Claude, ChatGPT, Grok, Z.ai, Kimi, DeepSeek. Did not include: Meta, Gemini, Wenxin, Qwen.
Source ↗

AutomationBench has two cuts. 68.1% counts finished objectives, while Meta's 37.8% counts a workflow only if it is fully done on the first try. Haiku drops from 35.4% to 13.9% the same way, so the two cuts can't be mixed.

Noticed by:
Included it: Claude, ChatGPT, Grok, Z.ai, DeepSeek. Did not include: Meta, Gemini, Kimi, Wenxin, Qwen.
Source ↗

Maker tables are the higher number. DeepSeek reports 42.3% on Terminal-Bench against 31.82% from Vals, and MiniMax reports 38.1% against 25.25%.

Noticed by:
Included it: Grok, Z.ai, Kimi. Did not include: Claude, ChatGPT, Meta, Gemini, DeepSeek, Wenxin, Qwen.
Source ↗

The harness is part of the score. Haiku's Terminal-Bench result is 39.2% from Anthropic, 36.36% from Vals and 32.8% from AA on the same 66 tasks.

Noticed by:
Included it: Meta, Grok, Z.ai, Kimi. Did not include: Claude, ChatGPT, Gemini, DeepSeek, Wenxin, Qwen.
Source ↗

FrontierCode 2.0's main score is a weighted rubric with blocking criteria, not just the share of patches that pass tests. Haiku scores 34.9% on AA's run and 27.7% on Cognition's own board.

Noticed by:
Included it: ChatGPT, Meta, Grok, Z.ai. Did not include: Claude, Gemini, Kimi, DeepSeek, Wenxin, Qwen.
Source ↗

04Questions people ask

What is the best model for an AI agent?

It depends on the job. Muse Ember is the pick for office steps (68.1% of AutomationBench objectives), and Claude Haiku 5.5 for code that must pass maintainers' tests (34.9% on FrontierCode 2.0).

Source ↗
AI models vs AI agents: what is the difference?

A model generates the next step; an agent wraps it in tools and a loop. That is why Terminal-Bench 4.0 scores model-and-agent pairs over 66 tasks, and the same model's score moves with the harness.

What is the best AI model for Hermes Agent?

On the Hermes Index snapshot, Haiku 5.5 scores 37.00 at about $0.49 a task and DeepSeek V4.1 Flash 36.91 at about $0.26. Ember is not listed.

Source ↗
Why do office-workflow scores differ so much between sources?

Artificial Analysis counts finished objectives (Ember 68.1%), while Meta's chart uses strict single-attempt scoring (Ember 37.8%, Luna 20.2% vs 53.2%).

Source ↗

05How we counted

In plain words, no formulas.

1The question

Top 10 Small AI Models for Agents: Office Workflows vs Code a Maintainer Would Merge

2Why this topic

Three independent publishers measure three different kinds of agent work, so the reader learns that 'beats Haiku' depends on the job, which no single leaderboard shows.

3Where it applies

Global, small and low-price models used in agents

4What we looked at

  • Share of business-workflow objectives finished inside simulated office apps
  • Share of coding patches that pass tests written by real repository maintainers
  • Share of real terminal tasks a model-and-agent pair resolves
  • How much each score moves between the maker's table and an independent run
  • Artificial Analysis AutomationBench-AA — Share of business-workflow objectives finished in simulated apps (Gmail, Slack, Salesforce and others). The objective-level score and the strict pass@1 score d
  • Cognition FrontierCode 2.0 (and Artificial Analysis's own run) — Whether patches written from informal specs pass maintainers' tests and rubrics, on 132 tasks
  • Laude Institute and Harbor Terminal-Bench 4.0 — Share of 66 containerized terminal tasks a model-and-agent pair resolves
  • Vals.ai Terminal-Bench 4.0 — The same 66 tasks run through one shared harness, used as an outside check on maker tables

5Who ranked

The leading AIs, each building its own list of ten independently and naming a source for every place: Claude, ChatGPT, Meta, Grok, Gemini, Z.ai, Kimi, DeepSeek, Wenxin, Qwen.

6How the places were set

The same entry written differently by different AIs was counted as one. The more AIs put an entry on their list, and the higher they put it, the higher its place. "AI agreement" shows how many of the AIs had it on their own list. Sources are only those the AIs found themselves; nothing was added by hand.

06Each AI's own list

🇺🇸 Claude10 places · 3 notes

The same question, answered by Claude alone. The final list above combines all the AIs.

  1. 1
    Meta Muse Ember (2026)

    Wins office work at 37.8% on AutomationBench-AA (Haiku 13.9%), but loses code: 27.2% vs 34.9% on FrontierCode 2.0.

    Source ↗
  2. 2
    Claude Haiku 5.5

    The code pick: best FrontierCode 2.0 score in the small group (34.9%), yet it finishes only 13.9% of office workflows.

    Source ↗
  3. 3
    Inkling 2 Small (Thinking Machines)

    Reported to jump from 5% to 46% on AutomationBench-AA over its predecessor, while Terminal-Bench 4.0 stays at about 13%.

    Source ↗
  4. 4
    DeepSeek V4.1 Flash

    Best long-session coder in Bito's group (71.9%) at $2.17 per 20-turn run; no AutomationBench-AA figure found.

    Source ↗
  5. 5
    MiniMax M3.1 Flash

    Reaches 75.0% on Bito's 20-turn sessions, nearly level with Haiku 5.5, but needs 3 to 5 hours per run.

    Source ↗
  6. 6
    GPT-6 Luna

    Middle on office work (20.2%) and last of the three launch rivals on code (25.2%) and terminal tasks (16.4%).

    Source ↗
  7. 7
    Muse Spark 1.3 (Meta)

    Scores 37 on the index, but the bill is $2.39 per task, and I found no workflow or code split for it.

    Source ↗
  8. 8
    Kimi K3.1 Pro

    A 64.9% coding score on Bito's tests, but at $29.32 per 20-turn run it is not the cheap choice.

    Source ↗
  9. 9
    GLM-5.3 (Z.ai)

    60.1% overall on Bito's coding sessions at $20.28 per 20-turn run; no office-workflow score found.

    Source ↗
  10. 10
    Claude Sonnet 5.5 (mid-price reference)

    Not a small model, but the 30% Terminal-Bench 4.0 score (medium effort) shows Inkling's 13% in context.

    Source ↗

What Claude noticed

Ember and Haiku 5.5 swap places by task type: Ember has 37.8% vs 13.9% on AutomationBench-AA, while Haiku has 34.9% vs 27.2% on FrontierCode 2.0. Both are Artificial Analysis runs.

AutomationBench-AA has two readings: 37.8% counts a workflow only if fully done first time, and 68.1% counts finished objectives.

On Artificial Analysis's Coding Agent Index, Ember scores 46 against Haiku's 37, so the FrontierCode loss does not carry over to every coding measure.

🇺🇸 ChatGPT10 places · 3 notes

The same question, answered by ChatGPT alone. The final list above combines all the AIs.

  1. 1
    Meta Muse Ember (2026)

    The office specialist: 68.1% workflow-objective completion, but only 27.2% on the mergeability-oriented coding evaluation.

    Source ↗
  2. 2
    Claude Haiku 5.5 (2026)

    It loses the office-workflow contest to Ember but beats it by 7.7 points on the reported FrontierCode comparison.

    Source ↗
  3. 3
    OpenAI GPT-6.1 Sol (2026)

    Its 39.0% FrontierCode score and 58.2% official terminal resolution justify escalating difficult patches to a stronger model.

    Source ↗
  4. 4
    Z.ai GLM-5.3-Flash (2026)

    35.8% terminal resolution for a Flash model, but only 14.2% on the maintainer-quality coding benchmark.

    Source ↗
  5. 5
    DeepSeek V4.1 Flash 1017 (2026)

    A 33-point intelligence score and open weights are appealing, but FrontierCode drops to 19.3% on harder merge-ready work.

    Source ↗
  6. 6
    OpenAI GPT-6 Luna (2026)

    A low-cost helper with 31 on the Coding Agent Index, but only 16.4% official terminal-task resolution.

    Source ↗
  7. 7
    Google Gemini 3.8 Flash (2026)

    A multimodal million-token assistant, yet only 19.1% of official terminal tasks were resolved in its evaluated harness.

    Source ↗
  8. 8
    Thinking Machines Inkling 2 Small (2026)

    Its 22.6% FrontierCode score is respectable, but a $1.18 index task and slow completion weaken the agent economics.

    Source ↗
  9. 9
    Cognition SWE-2 Flash (2026)

    A coding-focused Flash option reaches 24.0% on FrontierCode, but that alone does not establish office-agent reliability.

    Source ↗
  10. 10
    Meta Muse Spark Open (2026)

    Open weights and agent integration are useful, but 6.2% FrontierCode performance makes independent patch review essential.

    Source ↗

What ChatGPT noticed

Ember leads Haiku in business workflows, yet Haiku leads Ember in mergeability: 68.1% AutomationBench objective completion for Ember contrasts with Haiku's 34.9% versus Ember's 27.2% on FrontierCode-AA.

Cognition made FrontierCode 2.0 substantially harder: DeepSeek V4.1 Flash fell from 43.9% on version 1.1 to 19.3% on version 2.0. This is a benchmark revision, not a measured model regression.

A real maintainer's expectations are harder than passing ordinary tests: FrontierCode also checks scope, design and completeness, and even strong models can miss small cleanup requirements.

🇺🇸 Meta10 places · 4 notes

The same question, answered by Meta alone. The final list above combines all the AIs.

  1. 1
    GPT-6.1 Sol (2026)

    The only cheap-adjacent model here that is strong on both sides: 64.9% office, 39.0% merge-ready code for $1.03 a rollout.

    Source ↗
  2. 2
    MiniMax M3.1 Flash (2026)

    72.5% of office objectives finished - the best office score among cheap models - next to a 25% terminal score.

    Source ↗
  3. 3
    Gemini 3.8 Flash (2026)

    59.9% office, 19.7% terminal - the same split as MiniMax, at a far higher measured task cost.

    Source ↗
  4. 4
    Claude Haiku 5.5 (2026)

    The cheap code leader: 27.7% FrontierCode 2.0 and 36% terminal, but only 35.4% on office workflows.

    Source ↗
  5. 5
    Muse Ember (2026)

    Terminal-first: 40.9% Terminal-Bench in Meta's table, 27.2% FrontierCode-AA - office score not in the AA AutomationBench 24-model list found.

    Source ↗
  6. 6
    GPT-6 Luna (2026)

    Balanced-cheap: 53.2% office and 21.9% code - better office than Haiku, weaker code than Haiku.

    Source ↗
  7. 7
    GLM-5.3 (2026)

    62.2% office, 17.8% code - the widest office-over-code gap among the full-size GLM entries.

    Source ↗
  8. 8
    DeepSeek V4.1 Flash 1017 (2026)

    Terminal-strong for its price at 31.82% for $0.54, code-weak at 19.3% - a shell worker, not a merger.

  9. 9
    Kimi K3.1 Pro (2026)

    73.3% office - nominally the office leader - but the FrontierCode fall for the K3 line warns against reading it as a coder.

    Source ↗
  10. 10
    MiMo-V2.6-Flash (2026)

    Cheapest measured task here at $0.13, but no AutomationBench-AA, FrontierCode 2.0 or Terminal-Bench 4.0 row in the sources found - unproven on both jobs.

    Source ↗

What Meta noticed

Office and code diverge: MiniMax finishes 72.5% of office objectives but only about 26% of terminal tasks, while Haiku finishes 35.4% office but 27.7% code.

FrontierCode 2.0 cut every score: Opus fell 54.6% to 45.5%, Kimi K3 fell 60% to 17.7% - the new test separates tiers instead of clustering everyone in the fifties.

GPT-6.1 Sol delivers 86% of Opus's FrontierCode 2.0 score (39.0% vs 45.5%) at one-twelfth the rollout cost ($1.03 vs $12.82).

Haiku's office score depends on which AutomationBench you mean: 35.4% on AutomationBench-AA vs 12.0% on the original AutomationBench 1.0.6.

🇺🇸 Grok10 places · 4 notes

The same question, answered by Grok alone. The final list above combines all the AIs.

  1. 1
    Muse Ember (2026)

    Give it the office steps: 68.1% of workflow objectives, then keep it off the final merge.

    Source ↗
  2. 2
    Claude Haiku 5.5

    The cheap model that should touch the code: 34.9% merge-ready, 13.9% strict on the office test.

    Source ↗
  3. 3
    GLM-5.3-Flash

    An open model for the office queue, at 60.4%, with merge-ready code down at 17.6%.

    Source ↗
  4. 4
    Qwen3.8 Flash

    Office work at 55.9% sits 30 points above its terminal score on the same card.

    Source ↗
  5. 5
    DeepSeek V4.1 Flash 1017

    On the Hermes board it scores 36.91 for $0.26 a task, next to Haiku's 37.00 at $0.49.

    Source ↗
  6. 6
    Hy4 (Tencent)

    A terminal tie with Ember, 39.4%, bought at about six times Ember's test bill.

  7. 7
    MiniMax-M3.1-Flash

    The launch terminal number, 38.1%, and the shared-harness number, 25.25%, are not the same test.

    Source ↗
  8. 8
    MiMo-V2.6-Flash

    Index 27 at $0.13 a task, with 64 minutes on the clock and a launch terminal score that later read lower.

    Source ↗
  9. 9
    GPT-6 Luna

    Loses the office strict test at 20.2% and the merge test at 25.2%, on a sticker that matches Ember.

    Source ↗
  10. 10
    Gemini 3.8 Flash

    Knows more than it automates: 50.5% omniscience accuracy, at a $0.75 input price.

    Source ↗

What Grok noticed

The useful split is inside one price band. Muse Ember leads the office workflow rate that Artificial Analysis headlines at 68.1%, and Claude Haiku 5.5 leads merge-ready code at 34.9% against Ember's 27.2%. One model should take the office steps. The other should touch the patch.

GLM-5.3-Flash shows the same split without a second vendor: 60.4% on AutomationBench-AA and 17.6% on FrontierCode 2.0. A lead on finished workflows can sit beside a loss on code a maintainer would merge.

Haiku's terminal score is not one number. It is 39.2% on the launch card, 32.8% in Artificial Analysis's run, 36.36% on Vals, and 13.6% inside the Hermes Index snapshot. The harness is part of the result.

On the Hermes Index snapshot, DeepSeek V4.1 Flash scores 36.91 at $0.26 a task and Haiku 5.5 scores 37.00 at $0.49. Claude Opus 5.5 leads at 63.31, at a cost the mirror marks near $4.99. Everyday agent points are close. The bill is not.

🇺🇸 Gemini10 places · 1 notes

The same question, answered by Gemini alone. The final list above combines all the AIs.

  1. 1
    Anthropic Claude 3.5 Haiku

    Lags on raw SaaS app navigation (13.9%) but dominates small-model code quality with a 41.2% FrontierCode maintainer merge rate that rivals flagship models.

  2. 2
    Meta Muse Ember (2026)

    Crushes office workflows at 37.8%—nearly triple Haiku's score—but maintainers reject 72% of its code patches due to sloppy edge-case handling.

    Source ↗
  3. 3
    Qwen 2.5 Coder 32B-Instruct

    The open-weight coding specialist scores 36.6% on maintainer pull requests, beating every proprietary model in its price tier outside of Haiku.

    Source ↗
  4. 4
    Google Gemini 2.0 Flash

    The best balanced generalist, capturing 26.5% on office apps and 30.2% on terminal engineering without favoring either domain.

  5. 5
    OpenAI GPT-4o mini

    Executes standard office spreadsheets and CRMs reliably (20.2%) but stalls at 26.0% on FrontierCode when maintainers test asynchronous code paths.

    Source ↗
  6. 6
    DeepSeek-V3

    Solves complex algorithmic logic in pull requests (32.4% FrontierCode) but struggles with messy simulated OAuth and CRM token flows in office suites.

    Source ↗
  7. 7
    Meta Llama 3.3 70B Instruct

    Delivers balanced 28.5% coding and 27.3% terminal problem resolution, but its higher latency slows interactive CLI agent execution.

  8. 8
    Mistral Small 24B

    Excels at deterministic terminal scripting (24.2%) and ERP tool routing, but drops to 21.5% on maintainer merges due to non-idiomatic variable naming.

  9. 9
    Google Gemini 1.5 Flash

    Processes massive office inbox dumps without failing token limits, but maintainers merge fewer than 1 in 5 of its proposed codebase modifications.

  10. 10
    Nous Research Hermes 3 (Llama 3.1 70B)

    Fine-tuned specifically for agentic autonomy and function calling, achieving 22.7% on Terminal-Bench while office workflow rates hover at 14.2%.

What Gemini noticed

Specialized open-weight coding models like Qwen 2.5 Coder 32B beat generalist proprietary models (GPT-4o mini and Gemini 1.5 Flash) by over 10 percentage points on real repository maintainer reviews.

🇨🇳 Z.ai10 places · 2 notes

The same question, answered by Z.ai alone. The final list above combines all the AIs.

  1. 1
    Kimi K3.1 Pro (Moonshot AI, 2026)

    Runs the office (73.3% AutomationBench-AA) and the terminal (67.2%, the best TB4.0 score published) - but on merge-ready code its predecessor Kimi K3 managed only 17.7% on Cognition's board.

    Source ↗
  2. 2
    MiniMax M3.1 Flash (MiniMax, 2026)

    72.5% of office-workflow objectives - second only to Gemini 4 Carbon among 249 models - while Vals billed it just $0.21 per terminal test, the cheapest finished-task bill in the class.

    Source ↗
  3. 3
    Muse Ember (Meta, 2026)

    The office-code split in a single model: 68.1% on AA's workflow run (Meta's own stricter table says 37.8), a terminal win over Haiku 40.9% to 32.8% - and a loss to Haiku on merge-ready code 27.2% to 34.9%.

    Source ↗
  4. 4
    DeepSeek V4.1 Flash 1017 (DeepSeek, open weights, 2026)

    66.6% on office workflows and open MIT weights to self-host - but its terminal story flips with the harness: 42.3% self-reported versus 19.7% in Vals' independent run.

    Source ↗
  5. 5
    Claude Haiku 5.5 (Anthropic, 2026)

    The one to let touch the code: 34.9% of patches clear maintainers' tests on FrontierCode 2.0, about ten points over every cheap rival - while its office score (35.4%) is the weakest here.

    Source ↗
  6. 6
    GLM-5.3 (Z.ai, 2026)

    Budget-class best on Vals' terminal board (38.9%) plus 62.2% on office workflows - but Cognition has not graded its patches, so the mergeability question is open.

    Source ↗
  7. 7
    Gemini 3.8 Flash (Google, 2026)

    59.9% on office workflows with intelligence fans compare to Opus-tier - yet the same model cost Vals $8.65 per terminal test, the biggest finished-task bill in the budget class.

    Source ↗
  8. 8
    GPT-6 Luna (OpenAI, 2026)

    Its office score nearly triples depending on who scores it - 20.2% in Meta's single-attempt table versus 53.2% on AA's standard run - and mergeability sits at 23.8% on Cognition's board.

    Source ↗
  9. 9
    Qwen3.8-27B (Alibaba Qwen, open weights, 2026)

    The GUI specialist of the small tier: 84.3% on OSWorld-Verified (fourth overall) against 5.1% on Terminal-Bench 4.0 - desktop apps yes, containerized terminals no.

    Source ↗
  10. 10
    Kimi K3 (Moonshot AI, 2026)

    The cautionary tale: $3/$15 list pricing still produced 17.7% on Cognition's merge-ready board and $12.48 per terminal test in Vals' run - a price tag alone never bought agentic reliability.

    Source ↗

What Z.ai noticed

Desktop and terminal skills barely correlate in the small tier: Qwen3.8-27B is fourth overall on OSWorld-Verified (84.3%) but resolves 5.1% of Terminal-Bench 4.0 tasks - pick by where your steps actually run.

Merge-ready code is brutally hard at every price: even Claude Opus 5.5, FrontierCode 2.0's leader, passes only 49.1% of maintainers' tests, and no cheap model clears 35%.

🇨🇳 Kimi10 places · 3 notes

The same question, answered by Kimi alone. The final list above combines all the AIs.

  1. 1
    Claude Haiku 5.5 (2026)

    The only small model that wins the code column — FrontierCode 2.0 at 34.9% beats Muse Ember by 7.7 points — while taking 39.2% on Terminal-Bench.

    Source ↗
  2. 2
    Meta Muse Ember (2026)

    Finishes 37.8% of office workflows — 2.7x Haiku's rate — but the maintainer's-merge test flips it: 27.2% vs Haiku's 34.9%.

    Source ↗
  3. 3
    GLM-5.3 (2026)

    Best released-weights terminal score (43.4%) — an open model out-working every cheap closed rival at the command line.

    Source ↗
  4. 4
    DeepSeek V4.1 Flash 1017 (2026)

    Cheapest serious code option: 19.3% on FrontierCode 2.0 at $0.38 per rollout, with Terminal-Bench at 42.3% self-reported / 33.8% independently.

    Source ↗
  5. 5
    MiniMax M3.1 Flash (2026)

    38.1% on Terminal-Bench 4.0 — 22 points above Luna at the same ~$0.60 output price, with no office or merge-code numbers published yet.

    Source ↗
  6. 6
    GLM-5.3-Flash (2026)

    35.8% on Terminal-Bench 4.0 for $0.36 a task — the price-performance point where cheap terminal work starts being reliable.

    Source ↗
  7. 7
    GPT-6 Luna (2026)

    The office-over-code split in one model: strong on business workflows (AutomationBench-AA 53% in one AA run, 20.2% in Meta's stricter chart) but 16.4% on Terminal-Bench.

    Source ↗
  8. 8
    Gemini 3.5 Flash (2026)

    42.6% on AutomationBench-AA — the best office-workflow number any sub-frontier model posted on that board at launch.

    Source ↗
  9. 9
    MiMo-V2.6-Flash (2026)

    24.2% on Terminal-Bench 4.0 at $0.21 a task — cheapest terminal pass rate, but with a caveat: Vals found two-thirds of its coding tasks may leak the answer.

  10. 10
    Gemini 3.1 Flash-Lite (2026)

    Under $0.05 per office-workflow task — the floor of the AutomationBench-AA cost range, for steps where a retry is free.

    Source ↗

What Kimi noticed

The office/code split is real and large: Muse Ember beats Claude Haiku 5.5 by 24 points on business workflows (37.8 vs 13.9) and loses by 7.7 points on merge-ready code (27.2 vs 34.9) — one model for the inbox, another for the repo.

Maker tables inflate: DeepSeek V4.1 Flash 1017 scores 42.3% on Terminal-Bench 4.0 in DeepSeek's own evaluation but 33.8% when Artificial Analysis runs it — an 8.5-point drift in one direction only.

Open weights have closed the terminal gap but not the office gap: GLM-5.3 (43.4%) and DeepSeek V4.1 Flash (42.3%) lead released weights on Terminal-Bench 4.0, while no open model appears near the top of AutomationBench-AA.

🇨🇳 DeepSeek10 places · 2 notes

The same question, answered by DeepSeek alone. The final list above combines all the AIs.

  1. 1
    Muse Ember (Meta, 2026)

    The only cheap model that is simultaneously top-tier on business workflows (37.8% strict) and competitive on terminal work (40.9%), though only 27.2% of its coding patches would be merged.

    Source ↗
  2. 2
    Claude Haiku 5.5 (Anthropic, 2026)

    The merge-ready coder of the cheap class: 34.9% on FrontierCode 2.0, but only 13.9% strict on office workflows — it will fix the code and ignore the spreadsheet.

    Source ↗
  3. 3
    DeepSeek V4.1 Flash 1017 (DeepSeek, 2026)

    The terminal specialist: 42.3% on Terminal-Bench 4.0 and 60.5% on AutomationBench v1.0.6, but only 28% on FrontierCode — strong at running commands, weaker at shipping mergeable patches.

    Source ↗
  4. 4
    MiMo-V2.6-Pro (Xiaomi, 2026)

    A strong office performer at 59% AutomationBench-AA and 35% Terminal-Bench, but its 24% FrontierCode score means a maintainer is likely to send the patch back.

    Source ↗
  5. 5
    MiniMax M3.1 Flash (MiniMax, 2026)

    The office-workflow champion of the cheap set at 72.5% AutomationBench-AA, but only 21% FrontierCode and 26% Terminal-Bench — route it the spreadsheets, not the pull request.

    Source ↗
  6. 6
    GPT-6 Luna (OpenAI, 2026)

    A low per-task bill ($0.15) hides weak office and terminal results: 20.2% AutomationBench strict, 16.4% Terminal-Bench, and 25.2% FrontierCode.

    Source ↗
  7. 7
    GLM-5.3-Flash (Z.AI, 2026)

    A 60% AutomationBench score makes it a credible office step-runner, but 15% FrontierCode and 33% Terminal-Bench keep it away from repository work.

    Source ↗
  8. 8
    Ling 3.0 Flash (InclusionAI, 2026)

    At roughly $0.054 per task it is the cheapest office-capable model in this group, with 53% AutomationBench, but only 11–15% FrontierCode and 32.4% Terminal-Bench.

    Source ↗
  9. 9
    Gemini 3.8 Flash (Google, 2026)

    A 60% AutomationBench score at high effort, but only 19% FrontierCode and 20% Terminal-Bench — a reminder that office competence does not transfer to merge-ready code.

    Source ↗
  10. 10
    Nanbeige4.3-33B (Nanbeige, 2026)

    A small open-weight model that beats Qwen3.8 27B on AutomationBench (53% vs 48%) but drops to 11% on FrontierCode — a useful office helper, not a maintainer's merge candidate.

    Source ↗

What DeepSeek noticed

Muse Ember leads the cheap office-workflow table at 37.8% strict AutomationBench, but its 27.2% FrontierCode score shows the split clearly: the model that finishes a 30-step business process is not the model a maintainer wants reviewing a pull request.

MiniMax M3.1 Flash scores 72.5% on AutomationBench-AA — higher than Muse Ember — yet only 21% on FrontierCode. Office workflows and merge-ready code are genuinely separate skills in the cheap tier.

🇨🇳 Wenxin10 places · 1 notes

The same question, answered by Wenxin alone. The final list above combines all the AIs.

  1. 1
    Anthropic Claude Haiku 4.5 (2026)

    Leads office workflows and passes code tests — the only small model that does both well

  2. 2
    OpenAI GPT-4o mini (2026)

    Best small-model terminal finisher — but office steps and merge-ready code trail Haiku

    Source ↗
  3. 3
    Google Gemini 2.5 Flash-Lite (2026)

    Office workflow king of the cheap tier — code merge rate barely moves the needle

    Source ↗
  4. 4
    Meta Muse Ember (2026)

    Good enough on office, weak on merge-ready code — the 30-step job is its sweet spot

  5. 5
    DeepSeek-V3 Lite (2026)

    Code patches pass more often than its office share — the opposite of most small models

  6. 6
    Mistral Small 3.1 (2026)

    Solid bilingual office model — code merge rate sits below 1 in 4 patches

    Source ↗
  7. 7
    Cohere Command R+ 08-2026 (small)

    Office and RAG workflows lead — code and terminal are afterthoughts

    Source ↗
  8. 8
    Qwen 3 4B (2026)

    Terminal tasks done cheaply — but office workflows and code merge both fall short

  9. 9
    Microsoft Phi-4 Mini (2026)

    The smallest brain in the list — office, code and terminal all under 25%

  10. 10
    Stability AI Stable LM 2 1.6B (2026)

    Office workflows at 28% and code at 11% — cheap, yes; ready for agents, no

What Wenxin noticed

No small model passes more than 38% of FrontierCode 2.0 maintainer tests; the best (Haiku 4.5 at 38%) still fails 6 in 10 merge-ready patches — showing 'small' still means 'draft, not merge'

🇨🇳 Qwen10 places · 2 notes

The same question, answered by Qwen alone. The final list above combines all the AIs.

  1. 1
    Google Gemini 4 Argon

    Leads the SaaS automation charts, but stalls on real maintainer code reviews.

    Source ↗
  2. 2
    Meta Muse Ember

    Built specifically for the repetitive API calls and routing agents do all day.

    Source ↗
  3. 3
    Alibaba Qwen 2.5 Coder 32B

    The open-source king of terminal tasks and mergeable patches.

    Source ↗
  4. 4
    Anthropic Claude Haiku 5.5

    Anthropic's safety guardrails keep it from violating SaaS workflow rules.

  5. 5
    OpenAI GPT-4o-mini

    A reliable fallback for terminal commands, but weak on complex merge criteria.

    Source ↗
  6. 6
    Meta Llama 3.3 70B / 3.1 8B

    The default local choice for terminal chores, lagging on SaaS orchestration.

    Source ↗
  7. 7
    Mistral Large 4 / Small

    Punches above its weight on FrontierCode's strict grading rubric.

    Source ↗
  8. 8
    Alibaba Qwen 2.5 7B

    A surprisingly capable office assistant when constrained to Zapier-style workflows.

  9. 9
    DeepSeek Coder V2 / V3-Lite

    Strong at terminal automation, but struggles with multi-app business logic.

    Source ↗
  10. 10
    Google Gemma 2 9B

    Safe for lightweight local browser tasks, but easily confused by complex specs.

What Qwen noticed

AutomationBench-AA and FrontierCode 2.0 measure opposite skills; Gemini 4 Argon crushes office workflows (77.5%) but scores only 26.3% on real maintainer code merges.

Terminal-Bench 4.0 favors command-line native models, where Qwen 2.5 Coder 32B hits 65.2%, proving that terminal agents need different training than GUI agents.