Ollama: run open LLMs on your own machine, no cloud bill

One open-source tool a week. What it does, what it replaces, and something you can build with it — in under three minutes.

What it is

Ollama is a command-line tool and local server that downloads, runs, and serves open large language models — Llama, Gemma, Qwen, DeepSeek, gpt-oss and others — on your own laptop or server, with one command and no GPU cluster required.

What it replaces

For prototyping, internal tools, and anything where you don’t want prompts and data leaving your machine, Ollama replaces calling a hosted LLM API (OpenAI, Anthropic, etc.) for every test request, and it replaces hand-rolling a llama.cpp build-and-serve setup yourself. If you’ve ever written a bash script to download GGUF weights and wire up an inference server, this is that script, already built, maintained, and packaged for macOS, Windows, Linux and Docker.

What you can build

A private, local chat endpoint you can hit like any other API — useful for a CLI tool, a Slack bot, or a code-review script that never sends your source to a third party.

Install and pull a model:

curl -fsSL https://ollama.com/install.sh | sh
ollama run gemma4

That drops you into an interactive chat with Gemma running locally. For programmatic use, Ollama exposes a REST API on localhost:11434 — here’s a one-off request from a script:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{ "role": "user", "content": "Summarise this diff in one sentence: <paste diff>" }],
  "stream": false
}'

Wrap that curl call in a git pre-commit hook or a small Python script using the ollama-python library and you have a local commit-message or code-review assistant that runs entirely offline.

Where to get it

• Repo: github.com/ollama/ollama — 178k GitHub stars at time of writing
• Docs: docs.ollama.com (API reference migrating there from the repo’s docs/api.md)
• Licence: MIT — genuinely permissive, verified against the LICENSE file in the repo. Note the separate desktop GUI app has had community questions raised about its own licensing terms (see issue #11634) — the core CLI/server and the official Python and JS client libraries are all MIT.
• Maturity: production-ready for local/dev use and increasingly for self-hosted internal tools — this isn’t a weekend toy, it’s the de facto standard for running open models locally, with official Docker images and integrations for Claude Code, Codex, Copilot CLI and more.

One thing we couldn’t verify independently: Ollama’s own performance claims relative to raw llama.cpp aren’t broken out with benchmarks in the docs we reviewed — if you’re choosing between the two for a latency-sensitive workload, test it on your own hardware rather than taking anyone’s word for it.

---

Get next week’s tool by email. One tool, every week. Three minutes. No news roundups, no digest. — sign-up coming shortly