AI accounting test: open models and GPT-5.6 on a real Excel task
Benjamin Spinola, Founder
Published Updated 7 minutes read
This AI accounting test gave current models a real task from OpenAI's GDPval benchmark: read 66 invoices and build an Excel workbook that spreads prepaid expenses over the months they cover and reconciles them to the general ledger. The open model GLM-5.3 Flash delivered the best workbook, with 90 of 95 points, ahead of OpenAI's GPT-5.6 Sol, which scored 82 points in its first run and 72 in its second.
No workbook was free of errors. The highest scores concealed wrong balances and false statements that only a line-by-line check revealed, which is why this page reports the errors alongside the points.
Results
Each row is one model in one setting.
| Model | Points | Time | Cost | Assessment |
|---|---|---|---|---|
| GLM-5.3 Flash, open weightsHosted by Z.ai | 90 of 95 | 29 minutes | $0.12 | It passed 55 of 56 criteria; only the layout fell short, because account descriptions overlap neighbouring rows. An unscored note wrongly states that an insurance invoice was paid on 15 January 2025, whereas the source gives only a due date. |
| GPT-5.6 Sol, first runHosted by OpenAI | 82 of 95 | Not valid | At least $1.14 | It passed 51 of 56 criteria. The sheet names omit the account numbers, the end date of the insurance policy is missing, a difference of half a cent remains, and one cell displays ##### instead of its value. |
| Qwen3.8-Flash-Next, open weightsA rented machine with the NVIDIA GB10 chip of the DGX Spark | 80 of 95 | 33 minutes | $1.08 machine rental | It passed 49 of 56 criteria, but 46 of 264 historical invoice balances are wrong, because future invoices were counted too early. All eight ledger reconciliation checks nevertheless matched, so the totals concealed the errors. |
| DeepSeek V4.1 Flash, open weightsHosted by DeepSeek | 78 of 95 | 9 minutes | $0.14 | It passed 49 of 56 criteria. The start and end dates of the invoices are missing, 42 invoice rows differ from the totals by one cent, column headings are cut off, and an unscored note wrongly ends the amortisation of services in June. |
| GPT-5.6 Sol, second runHosted by OpenAI | 72 of 95 | 7 minutes | $1.46 | With identical settings, it passed only 46 of 56 criteria: 198 per-invoice balances for January to March are missing, and the sheet names, dates and display have faults. |
| Qwen3.8-27B, open weights, reasoning switched offA rented ASUS Ascent GX10 with the NVIDIA GB10 chip | 39, 35 and 9 of 95 | 30 minutes each | About $0.97 machine rental for all three | All three attempts stopped at the 30-minute limit. The workbooks stored calculations as text, booked future invoices too early or contained no figures at all. |
For hosted models, the cost is the charge for using the model; for the two Qwen models, it is the rental of the machine. The time of GPT-5.6 Sol's first run is not valid, because the computer that ran the test went to sleep during the attempt, and its cost is incomplete for the same reason.
The task
The task comes from GDPval, OpenAI's public set of 220 tasks drawn from the real work of 44 occupations. It asks the question that a finance team or an accounting firm faces at every month-end: can AI prepare the prepaid-expense schedules from the invoices?
- The input consists of six files: a chart of accounts in Excel, four monthly invoice files for January to April 2025 and one file of insurance invoices, 66 PDF pages in total.
- The model has to read all 66 invoices, assign each one to the account for prepaid services or for prepaid insurance, and spread each cost over the months that it covers.
- The deliverable is an Excel workbook with three sheets: a schedule for every invoice and a monthly reconciliation to the general-ledger balances up to April 2025.
- The task's original rubric checks 56 criteria worth 95 points, from correct amounts and working formulas to the presentation of the sheets.
How we ran the test
Every model received the same task, the same files, the same tools and the same instructions, and nothing was corrected for it during or after the attempt.
- The agent program Hermes gave each model the same tools in an isolated container: Python with spreadsheet and document libraries, LibreOffice to recalculate formulas, and Poppler to read the PDF invoices.
- Each attempt had at most 60 minutes, 80 steps and 65,536 tokens per response. The three Qwen3.8-27B attempts without reasoning had 30 minutes, 40 steps and 32,768 tokens, and DeepSeek V4.1 Flash had 32,768 tokens per response.
- The hosted models ran through the Vercel AI Gateway, each fixed to its maker's own service. The two Qwen models ran on rented machines with the NVIDIA GB10 chip, the chip of the DGX Spark.
- Each finished workbook was checked against every criterion of the original rubric, with an independent recalculation of the source figures and a rendered view of every sheet.
- The attempts took place between 15 and 21 September 2026.
What the result means for your decision
For a company that is considering AI on a machine of its own, the test supports four conclusions.
- Open models can deliver real accounting work: the best of them, GLM-5.3 Flash, delivered a better workbook than GPT-5.6 Sol at about a tenth of the cost per attempt.
- No model delivered a workbook that could be used without review. Even the best scores concealed wrong balances or false statements, so the output needs an accountant's check.
- Results vary between attempts: GPT-5.6 Sol scored 82 and then 72 points with identical settings, so a single run says little, and your own task should be tested more than once.
- The result closest to a machine of your own is Qwen3.8-Flash-Next on the GB10 chip of the DGX Spark, with 80 of 95 points. The comparison with ChatGPT and the AI model benchmarks page place it in a wider context.
Further reading: the comparison with ChatGPT covers seven office tasks, and the AI model benchmarks page collects the public ratings of the same models.
Questions about the AI accounting test
Can AI do accounting work in Excel?
Partly. In our AI accounting test on one real GDPval task, the best model built a reconciled Excel workbook from 66 invoices and scored 90 of 95 points, but every workbook still contained errors, from wrong balances to false statements. AI can prepare such work; an accountant still has to check it.
Which model did best in the test?
The open model GLM-5.3 Flash, with 90 of 95 points in its second attempt, followed by GPT-5.6 Sol with 82 points in its first run and Qwen3.8-Flash-Next with 80 points on a machine with the GB10 chip of the DGX Spark. With one task and few attempts, this order is not a ranking.
Did the models run on local hardware?
Only the two Qwen models ran on a machine with the NVIDIA GB10 chip, the chip of the DGX Spark. GLM-5.3 Flash, DeepSeek V4.1 Flash and GPT-5.6 Sol ran on their makers' hosted services, so their results do not show how they would perform on a machine of your own.
Are these official GDPval scores?
No. We used one task from OpenAI's public GDPval set with its original rubric, but the review was our own, assisted by an AI model and not blind. Official GDPval results come from professionals who compare a model's work with that of a human expert without knowing which is which.
Can we run a similar test ourselves?
Yes. Our test drive gives you one or two hours on a prepared machine with an open model, so that you can run a comparable task with your own tools and judge the result yourself.
Test your own accounting task on a real machine
Choose a setup, connect the tool that you already use and run a representative task. Within one or two hours, you will know whether a machine of your own can do the work, before you buy one.
Request an endpoint