Local AI for text: working without leaks, and what it costs you

Someone pastes a draft contract into a cloud chat and asks it to tidy up the wording. It takes four seconds and the result is good. That is the moment the leak happens, and nobody in the room notices, because a leak does not look like anything.

This is not a scare piece. Cloud providers are not reading your contracts for fun. But "we do not train on your data" is a promise about intent, not a statement about location — and once the text is on someone else's disk, in someone else's jurisdiction, inside someone else's backup, you no longer decide what happens to it. For a shopping list, who cares. For a client file, that is the whole question.

"Local" means three different things

The word is used loosely, and the differences matter.

• Local interface, cloud brain. A desktop app, but every request travels to a server. Most "AI assistants" are this. It is the same exposure as a browser tab with nicer buttons.
• Local by default, cloud on demand. Works offline, but hands specific tasks to an external model. Fine, if you know exactly which tasks.
• Actually local. The model file sits on your disk, the processing happens on your processor, and pulling the network cable changes nothing.

The honest price of the third option

I build a tool in that third category, so let me be the one to list the drawbacks rather than leaving it to a reviewer.

A first download of about a gigabyte and a half. The model has to physically arrive on your machine once. After that, nothing.

Several gigabytes of RAM, permanently. A small model needs roughly 4 to 8 GB free. On a machine with 8 GB total, you will feel it.

A weaker brain. This is the real trade, and it is worth being blunt about. A model that fits on your laptop is not a match for one running on a rack of accelerators. Ask it to reason across a hundred-page document and it will disappoint you.

What saved the idea is that most text work does not need the big brain. Cleaning up something you just dictated, cutting a paragraph in half, fixing grammar, translating a message, rewriting a sentence to sound less blunt — small models are genuinely good at these now. The tasks where the cloud is still clearly better are also, usually, the tasks that do not involve confidential text.

The two-minute test

Do not take anyone's word for it, including mine. Install the tool, then turn off Wi-Fi and unplug the cable. Now use the feature you care about.

If it works with the network dead, the processing is on your machine. If it spins, errors, or quietly degrades, your text was going somewhere. There is no marketing copy that survives this test, and it takes two minutes.

One caveat so the test is fair: many local tools still check a licence or download their model on first run. Do the test on the second launch, after setup is finished. What you are testing is whether your text travels — not whether the app ever touches the internet at all.

Where this leaves you

If your work involves text that belongs to someone else — clients, patients, employees, an employer with an NDA — the calculation is simple, because the downside is not "slightly worse output", it is a disclosure you cannot undo.

If your text is your own, use whatever is best. Nobody needs a local model to write a birthday message.

FAQ

How do I verify a tool is really local?

Disconnect the network on the second launch and use it. Works — it is local. Fails — it was not.

Is a local model much weaker?

For editing text you already have, no. For long reasoning over big documents, yes, noticeably.

What hardware do I need?

About 4–8 GB of free RAM and a normal modern processor. A graphics card helps but is not required.