AI model benchmarks 2026: open models compared with GPT-6 and Claude
Benjamin Spinola, Founder
Published Updated 14 minutes read
AI model benchmarks disagree because each one measures something different: expert-graded work products, long software tasks, paid freelance projects, blind votes, a combined index, and legal, finance and tax work. This page puts seven public rating systems side by side for eight current models, four of them closed and four whose weights anyone can download.
Every figure comes from the rating system's own page, which is listed under Sources with the date we read it. Where a system has not rated a model, the cell says so. Nothing is estimated and no gap is filled.
Why this page is here: we run open models on an NVIDIA DGX Spark for people deciding whether a machine of their own would do their work. Public scores are where that decision starts, so this page also shows which of these models fit on one DGX Spark at all.
Eight models across seven rating systems
The three highlighted rows at the top are the open model families our setups run: Qwen, DeepSeek and GLM. Read along a row to see one model in every rating system, or down a column to compare the models in one of them. Higher is better in every column except price. The small print under a score names the reasoning setting the rating system tested and, where it publishes one, its margin of error; where it tested several settings, the table shows the highest one it lists.
The table is wider than your screen. Scroll it sideways; the model names stay in place.
| Model | GDPval | METR | Remote Labor Index | LMArena | Artificial Analysis | Vals AI | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OpenAI, expert graderstasks won or tied against professionalsSource 1Sep 25, 2025 | GDPval-AAElo score, run by Artificial AnalysisSource 3Sep 28, 2026 | Time horizontask length finished half the timeSource 4May 8, 2026 | Automation rateprojects a client would acceptSource 6Sep 28, 2026 | Text scorefrom blind preference votesSource 7Sep 25, 2026 | Intelligence Indexcombined scoreSource 8Sep 28, 2026 | Speedoutput tokens per secondSource 8Sep 28, 2026 | API priceUS dollars per million tokens, input / outputSource 8Sep 28, 2026 | Legal researchaccuracySource 9Sep 22, 2026 | FinanceaccuracySource 10Sep 23, 2026 | TaxaccuracySource 11Sep 23, 2026 | |
| Qwen3.8-27BAlibaba · open weightsreleased Aug 2026 | not rated | 1,409xhigh · ±26 | not rated | not rated | 1,438 | 34xhigh | 46.3xhigh | 0.50 / 3.00 | 36.06%xhigh | 48.55%xhigh | not rated |
| DeepSeek V4.1 FlashDeepSeek · open weightsreleased Sep 2026 | not rated | 1,600max | not rated | not rated | 1,477max | 39max | 220.8max | 0.30 / 1.20 | 41.35%high | 53.48%high | 62.46%high |
| GLM-5.3-FlashZ.ai · open weightsreleased Aug 2026 | not rated | 1,641±23 | not rated | not rated | 1,474 | 42 | 56.6 | 0.15 / 0.50 | 45.19%max | 57.85%max | not rated |
| GPT-6 AstraOpenAI · closedreleased Sep 2026 | not rated | 1,542max · ±25 | not rated | 20.83% | 1,478max | 53max | 63.9max | 10.00 / 50.00 | 39.42%max | 53.54%max | 63.34%max |
| GPT-5.6 SolOpenAI · closedreleased Jul 2026 | not rated | 1,588max · ±21 | not rated | not rated | 1,483xhigh | 47max | 95.5max | 4.00 / 20.00 | 48.08%max | 53.76%max | 67.95%max |
| gpt-oss-120bOpenAI · open weightsreleased Aug 2025 | not rated | 596high · ±24 | 45 min20 min to 86 minSource 5 | not rated | 1,352 | 12high | 236.0high | 0.15 / 0.595 | not rated | not rated | not rated |
| Claude Opus 5.5Anthropic · closedreleased Sep 2026 | not rated | 1,846max · ±23 | not rated | not rated | 1,509high | 58max | 95.5max | 4.00 / 20.00 | 50.48%max | 58.59%max | 70.50%max |
| Claude Fable 5.1Anthropic · closedreleased Sep 2026 | not rated | 1,735max · ±25 | not rated | 17.92% | 1,501max | 53max | 69.7max | 10.00 / 50.00 | 55.29%max | 58.88%max | 77.64%max |
Not rated means the rating system has not published a figure for that model. The Stanford AI Index has no column, because it does not score individual models.
Bars show each score against the highest in its column, the way a spreadsheet's data bars do. Price has none, because a higher price is not better, and neither has a column with a single rating.
Under each column heading are the number of its source in the list at the foot of the page and the date of its figures: the source's own update date where it states one, and otherwise the day we read it.
Speed and price are Artificial Analysis's figures for the model maker's own service, and for gpt-oss-120b the median across the services that offer it. They are not measurements on a DGX Spark.
How each rating system works
Seven rating systems, seven different questions. Knowing which question a score answers matters more than the score itself.
OpenAI; Artificial Analysis
GDPval and GDPval-AA
OpenAI built GDPval from real work in 44 occupations. Industry experts compare a model's deliverable, such as a spreadsheet, a slide deck or a memo, with a professional's without knowing which is which, and the result is the share of tasks where the model wins or ties. OpenAI's expert-graded results, published in September 2025, cover older models such as GPT-5 and Claude Opus 4.1. We found none for the models here, and OpenAI's own leaderboard is no longer active, so that column says not rated throughout.
OpenAI's leaderboard now points to GDPval-AA, which Artificial Analysis runs on the 220 public GDPval tasks. Each model works on the tasks with a command line and web access, and blind comparisons of the finished work produce a score that rises as a model wins more often [an Elo rating]. The scale is pinned by setting DeepSeek V4.1 Flash at its max setting to exactly 1,600. Artificial Analysis publishes a margin of 21 to 26 points either way for the other scores here [a 95 per cent confidence interval], so smaller differences are not a ranking.
METR
METR time horizon
METR measures how long a piece of work an AI agent can finish on its own. It first records how long skilled people need for each of its tasks, which range from a few seconds to many hours and are mostly programming, machine learning and computer security work. It then lets the agent attempt every task several times and finds the task length at which the agent succeeds half the time [the 50 per cent time horizon].
An example: gpt-oss-120b has a time horizon of about 45 min. On tasks that take a skilled person about that long, such as training a small program that sorts data into categories [a classifier], which METR lists at about 49 min, the model succeeds about half the time. It succeeds more often on shorter tasks and less often on longer ones. The figure describes how long the task takes a person, not how long the model works on it. METR last updated its measurements on May 8, 2026, before every other model here was released, and it says itself that it cannot measure every release. The gpt-oss-120b figure comes from METR's original task set [Time Horizon 1.0], with a 95 per cent confidence range of 20 min to 86 min.
Scale AI and the Center for AI Safety
Remote Labor Index
The Remote Labor Index takes real projects that freelancers were paid for, from game development to architecture and data analysis, and gives them to AI agents. Trained reviewers compare each result with the professional's accepted work, and the score is the share of projects a reasonable client would accept [the automation rate]. The projects took the professionals 28.9 hours on average.
The leaderboard lists two of the models here, GPT-6 Astra and Claude Fable 5.1. The other six have not been rated.
Arena (arena.ai, formerly lmarena.ai)
LMArena, now Arena
People put a question to two anonymous models, read both answers and vote for the one they prefer. The votes become a score for each model. It measures which answers people prefer in conversation, not whether a model finishes a piece of work.
The table uses the text leaderboard's default view, with the setting Arena calls style control. It was last updated on Sep 25, 2026, from 8,528,723 votes on 409 models, and it rates all eight models here.
Artificial Analysis
Artificial Analysis Intelligence Index
Artificial Analysis runs ten evaluations itself, among them GDPval-AA, Terminal-Bench 4.0 and Humanity's Last Exam, and combines them into one score per model [Intelligence Index version 4.3.2]. It also measures how fast each model writes its answer and lists its price per million tokens [a token is a word or part of a word].
Speed is measured, and price is listed, for the service each model page names: the maker's own for every model here except gpt-oss-120b, where Artificial Analysis gives the median across the services that offer it. Neither tells you anything about speed on a DGX Spark, which depends on the machine.
Stanford HAI
Stanford AI Index
The AI Index is an annual report, not a leaderboard. It collects data on research, performance, investment and policy, and draws the trends out of it.
The 2026 edition, published in April 2026, scores none of the models here itself, so it has no column in the table. Its findings on open and closed models are summarised further down.
Vals AI
Vals AI
Vals AI tests models on legal, finance and tax tasks of its own. The table uses its current benchmark in each area: Legal Research Bench, which is research across US law; Finance Agent v2, which covers the core tasks of a financial analyst; and Tax Agent Bench, which asks research-grade US corporate tax questions.
Scores are accuracy in per cent as Vals AI reports it. On Legal Research Bench an answer counts only if every required element is right. gpt-oss-120b is in none of the three, and Tax Agent Bench has not rated Qwen3.8-27B or GLM-5.3-Flash.
Which of these models fit on one DGX Spark
An NVIDIA DGX Spark has 128 GB of memory, shared by its processor and its graphics chip. A model runs on one only if its weights fit in that space with room left for the text it is working on. The four closed models cannot run on a machine of your own at all, because their makers do not publish the weights.
For the four open models, the table lists the maker's own weights and the smallest complete builds we found. Storing each weight in fewer bits [quantisation] makes a model smaller and can cost accuracy.
| Model | Fits on one DGX Spark? | Builds and their size |
|---|---|---|
| gpt-oss-120b | Yes |
|
| Qwen3.8-27B | Yes | |
| DeepSeek V4.1 Flash | No | |
| GLM-5.3-Flash | Only below 4 bits |
Sizes are the total of the weight files each repository lists on Hugging Face, in gigabytes of a billion bytes. We added them up; the repositories do not state the totals. The memory figure is NVIDIA's.
Builds that remove parts of a model or read its weights from disk while it runs are left out, because they are a different model or a different use of memory.
Our lab ran the Inferact build of Qwen3.8-27B on a 128 GB machine with NVIDIA's GB10 chip, the chip inside the DGX Spark, in September 2026.
Source 13read Sep 28, 2026Our own measurement, on one GDPval task
Separately from the rating systems, and labelled as ours: in September 2026 our lab ran one accounting task from OpenAI's public GDPval set, and the table below shows how three of the models on this page performed on it. The job is to build a prepaid-expense amortisation workbook from six source files containing 66 invoices. The models worked through their makers' hosted services with the same tools and instructions, and we reviewed each finished workbook against the task's original rubric of 56 criteria worth 95 points.
| Model | Points | Run | Date |
|---|---|---|---|
| GLM-5.3-Flash | 90 of 95 | second attempt | Sep 16, 2026 |
| GPT-5.6 Sol | 82 of 95 | first of two runs | Sep 17, 2026 |
| GPT-5.6 Sol | 72 of 95 | second of two runs | Sep 17, 2026 |
| DeepSeek V4.1 Flash | 78 of 95 | one run | Sep 16, 2026 |
GLM-5.3-Flash's first attempt stopped without a workbook when it used up its response allowance of 32,768 tokens; its score comes from a second attempt allowed 65,536 tokens per response. DeepSeek V4.1 Flash ran with the 32,768-token allowance.
The full report of the AI accounting test describes the task, every attempt, including those that failed, and the errors found in each workbook.
The wider picture, from the Stanford AI Index
The 2026 AI Index scores none of these models itself, but four of its findings help to read the table. Page numbers refer to the report's PDF.
- Open models are close behind, not level. In March 2026 the top closed model on Arena, Claude Opus 4.6 (1,503), led the top open-weight model, GLM-5 (1,454), by 49 points or 3.4%. In August 2024 the gap had been 0.5%. Page 76.
- The top is crowded. The best models of the four leading companies were fewer than 25 Arena points apart in March 2026, and the report suggests that cost, speed, reliability and fitness for a particular field may matter more as a result. Page 78.
- Benchmarks have flaws of their own. A review cited in the report found invalid questions in widely used benchmarks, at rates from 2% in MMLU Math to 42% in GSM8K. Page 71.
- Model makers disclose less. The average score in the Foundation Model Transparency Index fell from 58 in 2024 to 40 in 2025, and developers of open models tended to score better. Page 164.
Summary: open models compete on professional work
As of September 2026, open models score ahead of OpenAI's two current closed models, GPT-6 Astra and GPT-5.6 Sol, on two ratings of professional work. On GDPval-AA, which rates models on the 220 public GDPval tasks, GLM-5.3-Flash scores 1,641 against 1,588 for GPT-5.6 Sol and 1,542 for GPT-6 Astra, a lead larger than the published margins. On Vals AI's finance benchmark it reaches 57.85%, against 53.76% for GPT-5.6 Sol and 53.54% for GPT-6 Astra. In Arena's blind votes, DeepSeek V4.1 Flash scores 1,477, against 1,478 for GPT-6 Astra.
Anthropic's models still lead most of the ratings. Claude Opus 5.5 scores 1,846 on GDPval-AA, and on Artificial Analysis's Intelligence Index, which combines ten evaluations, it scores 58 against 42 for GLM-5.3-Flash.
Open models have one property that the closed ones lack: because their makers publish the weights, they run on hardware you own. Qwen3.8-27B and gpt-oss-120b fit on one DGX Spark at four bits or more, GLM-5.3-Flash fits only in builds below four bits, and DeepSeek V4.1 Flash needs more memory than one machine has. On a machine of your own, you decide where your documents and prompts are processed and who has access to them, instead of sending them to a model maker's service.
Whether an open model that fits on your own machine does your work well enough is best judged on your own tasks. We run a DGX Spark with an open model by the hour; you connect the tools you already use and run your own workflows on it before you decide on a machine of your own.
Sources
Every figure on this page comes from one of these pages, all opened on Sep 28, 2026. Where a page states when it was last updated, that date is given too.
- OpenAI: GDPval: measuring the performance of our models on real-world tasks, updated Sep 25, 2025, read Sep 28, 2026
- OpenAI: GDPval leaderboard, read Sep 28, 2026
- Artificial Analysis: GDPval-AA v2.1 Leaderboard, read Sep 28, 2026
- METR: Task-Completion Time Horizons of Frontier AI Models, updated May 8, 2026, read Sep 28, 2026
- METR: Time Horizon 1.0 results (benchmark_results_1_0.yaml), read Sep 28, 2026
- Scale AI: Remote Labor Index (RLI) leaderboard, read Sep 28, 2026
- Arena (formerly LMArena): Text Arena leaderboard, overall, default view with style control, updated Sep 25, 2026, read Sep 28, 2026
- Artificial Analysis: LLM Leaderboard and model pages, Intelligence Index v4.3.2, read Sep 28, 2026
- Vals AI: Legal Research Bench, updated Sep 22, 2026, read Sep 28, 2026
- Vals AI: Finance Agent v2, updated Sep 23, 2026, read Sep 28, 2026
- Vals AI: Tax Agent Bench, updated Sep 23, 2026, read Sep 28, 2026
- Stanford HAI: The 2026 AI Index Report (April 2026), read Sep 28, 2026
- NVIDIA: NVIDIA DGX Spark, read Sep 28, 2026
- OpenAI on Hugging Face: openai/gpt-oss-120b, read Sep 28, 2026
- Qwen on Hugging Face: Qwen/Qwen3.8-27B, read Sep 28, 2026
- Inferact on Hugging Face: Inferact/Qwen3.8-27B-NVFP4, read Sep 28, 2026
- DeepSeek on Hugging Face: deepseek-ai/DeepSeek-V4.1-Flash, read Sep 28, 2026
- NVIDIA on Hugging Face: nvidia/DeepSeek-V4.1-Flash-NVFP4, read Sep 28, 2026
- apetersson on Hugging Face: apetersson/DeepSeek-V4.1-Flash-MixedQ2-GGUF, read Sep 28, 2026
- Z.ai on Hugging Face: zai-org/GLM-5.3-Flash, read Sep 28, 2026
- NVIDIA on Hugging Face: nvidia/GLM-5.3-Flash-NVFP4, read Sep 28, 2026
- Unsloth on Hugging Face: unsloth/GLM-5.3-Flash-GGUF, read Sep 28, 2026
Our own measurement comes from our lab records of the runs in its table. Its task is listed under the identifier 7d7fc9a7-21a7-4b83-906f-416dea5ad04f in OpenAI's public GDPval set, together with its source files and rubric.
More to read: the comparisons of private-local.ai with ChatGPT, Palantir, LocalAI and Private AI, and what local AI is and when it makes sense.
Questions about AI model benchmarks
Which AI model benchmarks should I trust?
Trust each one for what it measures, and none of them for everything. GDPval and the Remote Labor Index look at finished work products, METR at how long a task an agent can complete, Arena at which answers people prefer, and Vals AI at legal, finance and tax work. Start with the one closest to your own work, then test the model on your own tasks, because no public benchmark was built from them.
Why does a model do well in one rating system and poorly in another?
Because the systems test different work under different conditions: other tasks, other tools, other reasoning settings and other judges. In this table, GPT-6 Astra has the highest Remote Labor Index score of the models listed but a lower GDPval-AA score than GPT-5.6 Sol. Both results can be right at the same time.
Why do so many cells say not rated?
Rating systems test new models on their own schedules, and seven of these eight models were released between July and September 2026. METR has not published new measurements since May 2026, OpenAI no longer runs its own GDPval leaderboard, and the Remote Labor Index has rated two of the eight models so far. We leave those cells empty rather than guess.
Can I run these models on a machine of my own?
Only the open ones. gpt-oss-120b and Qwen3.8-27B fit on one DGX Spark with 128 GB of memory. GLM-5.3-Flash fits only in builds that store each weight in fewer than four bits, and we found no complete build of DeepSeek V4.1 Flash that fits at all. The closed models from OpenAI and Anthropic cannot run on your own hardware.
How close are open models to the closed ones?
It depends on the rating. On Arena, DeepSeek V4.1 Flash and GLM-5.3-Flash score close to GPT-6 Astra and GPT-5.6 Sol, and on GDPval-AA GLM-5.3-Flash scores above both. On the Intelligence Index, every closed model in the table scores above every open one. The Stanford AI Index found that the gap between the best closed and the best open model on Arena opened again in 2025, after nearly closing in 2024.
Public scores start the question. Your own tasks answer it.
Pick a setup, connect the tools you already use and run the work you care about on the model and machine you would buy. In an hour or two you know whether it does the job.
Request an endpoint