What a local AI model can and cannot do for a small business
Most AI assistants run in the cloud: what you type goes to the provider's servers. A local model runs on your own computer or server instead. Open models such as Llama (Meta) or the open models of Mistral can be run that way, for example with the free tool Ollama.
That sounds like the answer to every privacy worry. It is a good answer to some of them. This is what to expect.
What it does well
Well-defined tasks on one document at a time: summarising, sorting, and pulling fixed fields out of a document such as a delivery note or an order form.
And the data stays where it is. For a company that handles other people's plans, files or contracts, that can be the whole point.
Where it falls short
The models that fit on ordinary office hardware are smaller than the largest cloud models. You notice that in tasks that need reasoning across several documents at once, and in text that goes to customers, where tone matters.
Like every language model, a local one can also give a wrong answer with full confidence. A human check stays necessary.
What it costs
No subscription, but not free. Count three things.
- Hardware: a machine with enough memory and preferably a strong graphics card. The bigger the model, the more it needs.
- Setup: installing the model, connecting it to your documents or workflows, and testing.
- Maintenance: updates, new model versions and someone who looks at it when it stops. Without an owner it dies quietly.
When it is worth it
When there is a real reason why documents may not leave the building, and the task is narrow and repetitive.
If the worry is mainly 'will they train on our data?', look first at the business plans of the cloud assistants. Several of them do not train on business data by default, and that is far less work than running your own model.
Three tips to start
- Pick one task, not 'a general assistant'. A local model is a tool for one job.
- Test on real documents from the past, and compare the result with what a person would have filled in.
- Decide in advance who maintains it and how many hours that may take.
See where the gains are.
Ready for your first small step?
Book an intro call with Senne. No strings attached.
Either way you leave with a few concrete pointers.

