Local AI: what it is, how it works and when it pays off

Local AI describes where a model runs, not who developed it. An open model from a large AI company that runs on your own machine is local AI, whereas the same open model running on a cloud provider's servers is cloud AI. Local AI also normally means running a model that has already been trained, known as inference, rather than training a model from scratch. Running a model on your own machine has three practical consequences.

• The model files are stored on your own disk. You can fix an exact version and keep it working in the same way next year, which a provider's hosted model does not guarantee.
• On a machine that you own, no data leaves the building for processing. This is the main reason why companies consider local AI.
• Once the software and the model are installed, the machine answers without an internet connection.

Running a model locally takes three steps.

1. You install software that serves the model, such as Ollama, LM Studio or LocalAI. Most of these accept the same request format as OpenAI, so an existing tool usually needs only a new address.
2. You download an open model in a size that fits the memory of your machine. The model Qwen3.5-35B-A3B, with 35 billion parameters, needs 64.6 gigabytes in full precision and 12.2 gigabytes in a compressed version, so either fits on an NVIDIA DGX Spark with 128 gigabytes of memory.
3. The model processes each prompt on your own processor or graphics card and returns the answer without calling a cloud service. On a DGX Spark, the compressed Qwen model produces 74.4 tokens per second, which is roughly 55 words per second.

The guide how to run AI locally explains each step, the hardware requirements and how to judge the results.

Local AI is used for the same work as cloud AI, but companies choose it where the material is sensitive or where the same task recurs in large volumes. The following examples show typical uses, with published results for open models where they exist.

• A manufacturer has a model answer questions about its technical documentation and maintenance manuals, so that service staff find the right procedure without searching through thousands of pages.
• A law or tax firm has a model extract parties, deadlines and amounts from contracts and client correspondence and draft first summaries for review.
• An automation agency connects a local model to workflow software such as n8n, so that incoming emails are sorted and answered automatically. On AutomationBench-AA, which simulates such workflows across Gmail, Slack and Salesforce, the open model DeepSeek V4.1 Flash completed 68.9 per cent of the objectives, against 68.5 per cent for GPT-6 Astra; the comparison with ChatGPT covers seven such tasks.
• A finance team has a model analyse spreadsheets, comma-separated values (CSV) exports and logs where they are stored. In the DSGym study, the open model gpt-oss-120b answered 71 per cent of simple data-analysis questions correctly, against 75 per cent for GPT-5. In our own AI accounting test, the open model GLM-5.3 Flash built a reconciled Excel workbook from 66 invoices and scored 90 of 95 points.
• A clinic transcribes dictated reports and drafts discharge letters on its own hardware, so that patient data does not reach an external service. The comparison with the redaction company Limina lists seven such situations by industry.

For a company, local AI has four advantages over cloud AI.

• The company keeps control over its data and its intellectual property: contracts, patient records or source code are processed inside its own network. The comparison with Limina shows seven situations in which this is decisive.
• There are no fees per request or per person. A team of 20 on ChatGPT Business pays $6,000 a year, every year, whereas a machine is paid for once; the comparison with ChatGPT lists the current prices.
• The machine works on the company's own premises and without an internet connection, for example in a production network that is deliberately cut off from the internet.
• The company fixes the model and its version, for example a specific release of Qwen, DeepSeek or GLM, so that a result can be repeated next year in exactly the same way. A hosted model can change without notice.

An open model that fits on a desktop machine is less capable than the best cloud models, and no adjustment closes that gap entirely. The comparison with ChatGPT measures the gap task by task, and the AI model benchmarks page collects the public ratings.

• Cloud models remain clearly ahead on web research and image generation. On BrowseComp, a test of finding hard-to-locate facts on the web, GPT-6 Astra scores 91.5 per cent, whereas the best open models that fit on a single machine score 64 to 68 per cent.
• The memory limits the size of the model, and that limit is fixed on the day of purchase. A DGX Spark with 128 gigabytes holds compressed models of several tens of billions of parameters, but not the largest open models, which need several hundred gigabytes.
• Somebody has to install, update and repair the machine, and that person's time is usually a larger cost than the hardware.
• When several people send requests at the same time, they share the machine and each of them waits longer, so a machine for a team must be tested with several users at once.
• A machine that stands idle for most of the day is difficult to justify against a subscription of $20 per person per month for ChatGPT Plus.

The purchase price is the figure that is quoted most often, but it is the smallest part of the answer. Four costs determine whether a local setup pays for itself.

• The machine itself, for example an NVIDIA DGX Spark, is a capital purchase of several thousand dollars, paid once and depreciated like any other equipment.
• Power and space are the second cost. The machine consumes power while it is in use and much less while it is idle, and it needs a location with adequate ventilation rather than a closed cupboard.
• Maintenance is the third cost: updates, model changes and the occasional failure. Business cases most often omit this cost, and it is the one most likely to be underestimated.
• The current alternative sets the standard. ChatGPT Business costs $25 per person per month, so 20 users cost $6,000 a year; that is the figure that the machine has to beat, and it should be calculated before anything else.

Most of this question can be answered on paper before any hardware is used. Take one real task rather than a category of tasks, and check it against the following four questions.

• How often does the task run? A task that runs hundreds of times a day, such as sorting incoming emails, makes owning equipment worthwhile; a report written once a quarter does not.
• How good does the output need to be, and who checks it? Define the standard before you see any answers, for example with ten to twenty cases whose correct result you already know; the guide on running AI locally explains how.
• Is there a reason why the material cannot leave the building, such as professional secrecy, patient data or trade secrets? If there is, that reason usually decides the question on its own.
• Who will operate the machine? If nobody can be named, the company is not yet ready for one, whatever the benchmark results show.

Testing local AI properly normally requires owning the machine first. Our test drive exists to close that gap.

• We operate an NVIDIA DGX Spark with a named open model and provide a standard endpoint for your own tools, consisting of a web address and an access key.
• You connect the tool that you already use, for example an n8n workflow or an application built for OpenAI's interface, and run your own tasks for one or two hours at $10 per hour, as set out on the pricing page.
• You keep your measurements: how long each task took, what the model returned and where it was wrong.