My favourite MCPs: local-llm

Most of what I ask an AI to do is shovel work — sorting, summarising, pulling dates out of letters. This one hands that to a model running on my own machine.

Not sure what an MCP server is or how it’s used? Check out my explainer here.

local-llm gives Claude six specific jobs it can hand to a model running on my own computer: summarise, draft, classify, extract, transform, complete. Claude stays in charge and decides what to delegate. A small local model does the repetitive part. I built this one, because I kept watching an expensive model do work that needed no intelligence at all.

The model runs under Ollama, which is the easiest way to run a language model on your own machine — one command to install one, and it serves locally like any other bit of software.

What changed for me is which jobs are worth automating at all. Sorting every subject line in my inbox by type. Summarising every article I clipped this week. Pulling the dates and amounts out of a stack of letters. None of those need frontier reasoning, and all of them are per-item work, so the cost of doing them with a cloud model made automating them faintly ridiculous. I did them by hand or I didn’t do them. Now they run over every item, every day, and the marginal cost is electricity.

The part where people give themselves a bad time is hardware, so here are the real numbers. Mine runs on a headless Mac with 16 GB of memory that is also hosting a stack of containers, so there is nowhere near 16 GB free. One seven-billion-parameter model at Q4 — Q4 meaning the weights are compressed to roughly an eighth of full precision, barely perceptible on mechanical work and enormously smaller — comes to 4.7 GB and sits there comfortably. A 9.6 GB model plus the containers is 14.7 of 16 before a single question gets asked. Do that arithmetic before you download, not after.

And I say that having got it wrong myself. I had eight models on that machine for months, and my own documentation confidently stated the default was a Gemma model. It wasn’t. The config file pinned Qwen’s coder model and always had. The docs and the config had disagreed for months and I’d have sworn blind the docs were right. I deleted the six nothing was using, freed 32 GB, and nothing broke — because the model I believed was doing the work had never been doing the work. Then I checked the free space with df rather than trusting my own subtraction, which turned out to matter too.

Making it repeatable

The one-off ask is plain:

Classify these two hundred subject lines by type using the local model, then show me only the ones it couldn’t place.

The standing version lives in a skill — which is just a text file of instructions the assistant reads when the work comes up, written in English rather than code:

Routing rules for text work:
- Classify, summarise, extract, transform, translate → local model.
- Judgement, voice, security, anything client-facing → keep it
  with Claude.
- Check the config for which model is actually loaded. My own
  docs were wrong about this for months.
- One 7B model at a time. The machine has 16GB shared with
  everything else on it.
- Review what comes back. Local output is a first pass, not a
  finished one.

Five lines, and each one is a mistake I’ve already made — including the last, which is the one people skip. A small model will hand you a confident, tidy, wrong classification, and the whole arrangement only works because something reads it afterwards.

Setup is Ollama, one model pulled from their library, and a line of config to point the MCP server at it. The project is open source and MIT licensed.

The cloud model still does all the thinking here. It just stopped doing the sorting.