AI
On-premise AI
Free, open-source AI that runs on a computer you own, in your own building. No monthly fee per person, and your documents never leave the room.
An LLM, or large language model, is the engine behind ChatGPT, Claude and Copilot. Most are rented from the cloud. Some are published for anyone to download and run for free. These are called open-weight or open-source models.
“On-premise” means you run one of them on a computer in your own office. Staff open a chat page in their web browser, much like ChatGPT, but the thinking happens on that one machine.
One limitation: the free models you can run on an office PC are good, but not as clever as the biggest paid ones. Think of a very capable graduate, not your best senior specialist.
Six popular free models
These are six of the best-known open models that an office could realistically run. Model names change every few months, but the sizes and what they need stay much the same.
The best all-rounder for one good office PC. Writes, summarises, answers questions about your documents, and can read images.
Quick and capable for everyday work, and it understands pictures and screenshots as well as text.
OpenAI’s free model. Very fast on modest hardware and good at step-by-step reasoning and using tools.
Well suited to office assistants. Reliable at producing tidy, structured answers such as tables and forms.
Thinks through a problem before it answers. Good for maths and logic, but noticeably slower because of the extra thinking.
A big, well-known model. Too heavy for a normal office PC, and sluggish to use even on strong hardware.
The models cost nothing. The software to run them (Ollama and Open WebUI) is free too. Your real costs are three things: the computer, the electricity (roughly A$25 to A$40 a month for a machine that is on all day and busy part of it) and someone’s time to set it up and look after it.
Some organisations have a policy against models made in certain countries. That applies to Qwen and DeepSeek. Because they run in your office, your data does not go to their makers, but check your own policy before you choose.
What you need to run them
Forget processor speeds and spec sheets. What decides which AI you can run is the video memory on the graphics card. More memory lets you run a bigger, cleverer model.
Graphics card with 16 GB of video memory (such as a GeForce RTX 5070 Ti) and 32 GB of ordinary memory.
Cost: about A$5,000 for the complete machine.
Graphics card with 24 GB of video memory (such as a GeForce RTX 4090, often bought used) and 64 GB of ordinary memory.
Cost: about A$7,500 for the complete machine.
Graphics card with 32 GB of video memory (a GeForce RTX 5090) and 128 GB of ordinary memory.
Cost: about A$10,000. The RTX 5090 alone was about A$6,000 in Australia in March 2026.
What separates them: the graphics card and the ordinary memory. The rest of each machine (processor, fast solid-state drive, cooling, power-protection unit) is much the same. All prices are rough Australian-dollar estimates; graphics-card and memory prices have been rising, so check current prices.
The end result
A private chat page that lives on your office network. Staff open it in their browser, type a question and get an answer, with nothing sent outside the building.
A typical day: a manager drops a 40-page supplier contract into the chat, asks for the unusual clauses, and has a useful first read within a minute or so. A first read to check, not advice to rely on.
Ten people on one PC
It is a common belief that running your own AI saves money. For a team of ten it is no cheaper once you count everything, and it loses to Copilot.
| Option for 10 people | Per month | Over 3 years | Good to know |
|---|---|---|---|
| Copilot add-on to Business Standard | A$142 | A$5,112 | A$14.20 a person on top of Microsoft 365. Works inside Outlook, Teams and SharePoint. |
| Claude or ChatGPT business seats | A$280 to A$309 | A$10,080 to A$11,128 | About A$28 to A$31 a person. The cleverest models, nothing to look after. |
| Your own PC in the office | A$35power only | A$6,260 to A$11,260machine A$5,000 to A$10,000 plus power | Add someone’s time: two hours a month at A$150 an hour is another A$10,800 over three years, whichever machine you buy. |
Yes, for everyday use. Ten staff are rarely all asking at the same moment. Usually two to four are waiting on an answer at once. The free tool most people start with (Ollama) copes with about five at once. Past that you need a more capable setup, and it queues the rest.
The limit is memory. Every person working on a long document takes a slice of the card’s memory. On the smaller cards, several people summarising big files at once will slow everyone down.
Possible, but it is not a money-saver at this size. The figures above assume a machine between A$5,000 and A$10,000, A$0.30 a unit for power and A$150 an hour for IT time. Change them and the answer moves, but not by much for ten people.
The strong reason to do this is privacy and control, not cost. Cost only starts to favour your own machine with many more users, or heavy daily use that would run into usage limits on the paid tools.
Is it viable for my company?
Run through these. If most of the left column sounds like you, it is worth a trial.
Prices and figures are rough guides as at October 2026, in Australian dollars and ex GST, and will change. Model sizes and speeds come from public tests on RTX 4090-class cards: see the Ollama models cheat sheet, VRAM guide and Ollama vs vLLM comparison. The RTX 5090 price comes from a March 2026 Australian price review. Machine prices, power costs and IT time are our own assumptions.