Ollama Tutorial: How to Install and Run AI Models Locally

Ollama is a free, open-source tool that downloads AI language models and runs them on your own computer. To start, install it from the official Ollama download page, open a terminal, and type ollama run gemma4:e2b (or ollama run llama3.2:3b on a laptop with little memory). Ollama downloads the model and opens a chat; type /bye to leave, as the Ollama quickstart shows. After that, a handful of commands cover daily use: pull, ls, ps, stop and rm.

This tutorial is for curious beginners who want a private AI assistant that runs offline, and for anyone testing open models before building with them. Commands and requirements were checked against Ollama’s own documentation as of October 2026.

Related reading: Ollama vs LM Studio, how to run ChatGPT locally, and Obsidian AI. Key terms used in this guide: open-weights model, context window, and prompt injection.

What Ollama is and who this tutorial is for

Ollama has two parts. A background server does the work of loading a model into memory, and a command line tool called ollama talks to that server. On macOS and Windows there is also a desktop app: the Ollama blog announced on 30 July 2025 that the app lets you download models and chat with them in a window, including dropping in text files or PDFs.

The project’s code is published on GitHub under the MIT license. Running models locally costs nothing beyond your own hardware and electricity. The Ollama pricing page lists a Free plan at $0 that includes running models locally, and paid plans such as Pro at $20 per month that add usage credits for cloud models (verify current pricing on the official site).

A few terms you will see:

• Model: the AI file that generates text. Models differ in size, skills and license.
• Pull: download a model to your computer.
• Tag: the part after the colon in a name like gemma4:e2b, which picks a size or variant.
• Local: the model runs on your machine, so your prompts stay there.
• Cloud model: a model whose name ends in :cloud; the download page explains that these run on Ollama’s servers instead of your computer.

Step 1: Install Ollama on Mac, Windows or Linux

Before you install, check two things: your operating system version and your free disk space. Models are big. Ollama’s macOS notes warn that models can be tens to hundreds of GB in total, so plan room for more than one.

macOS

Ollama’s macOS requirements are macOS Sonoma (version 14) or newer. Apple M series Macs get CPU and GPU support; older Intel Macs run on the CPU only, which is slower.

1. Download the app from the official download page.
2. Open the .dmg file and drag Ollama into your Applications folder.
3. Start the app. On first launch it checks that the ollama command is available and asks permission to create a link in /usr/local/bin if it is missing.
4. Open Terminal and type ollama -v to confirm the version.

Windows

The Windows page lists Windows 10 version 22H2 or newer, Home or Pro. If you have an NVIDIA card, you need driver 551.61 or newer.

1. Download OllamaSetup.exe from the download page and run it. It installs in your user folder and does not need administrator rights.
2. Ollama then runs in the background, with a tray icon.
3. Open a new PowerShell or Command Prompt window and type ollama -v.

The installer needs at least 4 GB of space for the program itself, before any models.

Linux

The download page offers a one-line install script:

curl -fsSL https://ollama.com/install.sh | sh

This downloads a script and runs it, so if you prefer, save and read it first. If the background service did not start on its own, the quickstart says to launch it with ollama serve. Then confirm it works from a second terminal with ollama -v.

Hardware checklist before your first download. Ollama’s quickstart gives one concrete example: the Gemma 4 E2B model is about a 7.2 to 7.5 GB download, and Ollama recommends 8 GB of available graphics memory (VRAM), or unified memory on a Mac. With less, Ollama can fall back to regular system RAM, but responses may be slower. Use that as your yardstick:

• Know your memory: on a Mac, Apple menu, then About This Mac. On Windows, Task Manager, then Performance.
• Compare it with the model’s download size, listed on each model page in the Ollama library.
• Start small. You can always pull a larger model later.

Step 2: Pull and run your first model

With Ollama installed, one command is enough:

ollama run gemma4:e2b

If the model is not on your computer yet, Ollama pulls it first and shows a progress bar. When you see the >>> prompt, type a question and press Enter. To paste text that spans several lines, wrap it in triple quotes ("""), as the CLI reference shows. Type /bye to end the chat.

Try a worked example. Paste a long email you wrote yourself and ask: “Summarize this in three bullet points and list any dates mentioned.” Then follow up: “Rewrite the second bullet for a busy manager.” Summaries and rewrites of your own text are where small local models shine. They can also misread dates or invent details, so check anything that matters against the original.

Decision framework: which model to start with

Pick by memory first and quality second. A fast small model teaches you more than a large one that crawls.

Check the model license. Ollama is MIT licensed, but each model carries its own license. The library page for Gemma 4 E2B includes an Apache 2.0 license file, while Llama 3.2 ships with the Llama 3.2 Community License Agreement and an acceptable use policy. Licenses can limit commercial use or certain uses, so read the license section on the model page before you build anything for work or clients.

Basic Ollama commands cheat sheet

These commands come from the Ollama CLI reference and the Modelfile reference:

Inside a chat, you can change settings on the fly. The Ollama FAQ shows /set parameter num_ctx 4096, for example, which raises the context window: how much text the model can keep in view. A bigger window uses more memory.

A tidy habit: run ollama ls once a month and remove models you no longer use. Per the FAQ, models live in ~/.ollama/models on a Mac and C:\Users\%username%\.ollama\models on Windows. You can move them to another drive with the OLLAMA_MODELS environment variable.

Step 3: Use the local API, Python, a Modelfile and a chat window

Once a model runs in the terminal, you can reach it from other programs.

The local API and Python

Ollama serves an API at http://localhost:11434. The README on GitHub shows a request like this one, which you can paste into a Mac or Linux terminal:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:e2b",
  "messages": [{"role": "user", "content": "Give me three tips for a tidy desk."}],
  "stream": false
}'

The answer comes back as JSON. Chat apps, note tools and coding plugins use this same API.

The same README shows the official Python library. Install it with pip install ollama, then:

from ollama import chat

reply = chat(model="gemma4:e2b", messages=[
    {"role": "user", "content": "Explain what a CSV file is in two sentences."}
])
print(reply.message.content)

A practical first project: loop over a folder of your own notes and ask for a one-line summary of each, all on your computer.

Customize a model with a Modelfile

A Modelfile is a small text file that saves your favorite settings and instructions. The Modelfile reference lists the main instructions: FROM (required, the base model), PARAMETER, SYSTEM, TEMPLATE, MESSAGE and LICENSE. Here is a hypothetical example for summarizing meeting notes:

FROM gemma4:e2b
PARAMETER temperature 0.3
SYSTEM You summarize meeting notes in plain English. Use short bullet points and list action items with owners.

Save it as Modelfile, then run ollama create notes-helper -f Modelfile followed by ollama run notes-helper. A lower temperature makes answers steadier, which suits summaries.

Chat window options

If you prefer not to use a terminal, you have two routes. On macOS and Windows, the official Ollama app has its own chat window, and its settings include a context length slider. The GitHub README also lists community projects that connect to Ollama, such as Open WebUI for a browser interface and AnythingLLM for desktop use. These are third-party tools, not made by Ollama, so check who maintains them and what permissions they ask for before you install one.

Common mistakes, risks and troubleshooting

Most problems fall into a few patterns. The Ollama troubleshooting guide says logs are the best first step: on a Mac run cat ~/.ollama/logs/server.log, on Linux with systemd use journalctl -u ollama, and on Windows look in %LOCALAPPDATA%\Ollama for server.log.

• “ollama” command not found: on Windows the installer adds Ollama to your user PATH, so close the terminal and open a new one. On a Mac, start the app once so it can create its command link.
• Cannot connect to the server: the app or ollama serve is not running. Open the app, or start the server in a separate terminal.
• Answers are very slow: run ollama ps. The PROCESSOR column shows how much of the model sits on the GPU versus the CPU. If much of it is on the CPU, try a smaller model or close memory-heavy apps.
• Disk filling up: list models with ollama ls and delete extras with ollama rm.
• Strange squares in the Windows 10 terminal: the Windows notes say older terminal fonts may not show Ollama’s progress characters; change the font.
• Memory stays busy after you finish: the FAQ says models stay loaded for 5 minutes by default. Use ollama stop <model> to free memory at once.

Security: keep the API private

The most serious mistake is exposing the API. Ollama’s authentication page states that the local API at localhost:11434 does not require authentication. By default, the FAQ says, Ollama listens only on 127.0.0.1, meaning your own computer. Guides online sometimes suggest setting OLLAMA_HOST to 0.0.0.0 so other devices can connect. That opens the API to your whole network, and if your router forwards port 11434, to the internet, where anyone could use your machine to run models or delete them and, if you are signed in, run cloud models on your account and use up your credits.

Safer habits:

• Leave OLLAMA_HOST at its default unless you have a specific need.
• Never forward port 11434 on your router or open it in a cloud server’s firewall.
• If you need remote access, use a private VPN or a reverse proxy with its own login, set up by someone who manages servers.
• Allow only the browser extensions you trust through OLLAMA_ORIGINS, not every extension.

Privacy: local and cloud are different. The FAQ states that Ollama does not see your prompts when you run models locally. Cloud models are different: your prompts travel to Ollama’s servers for processing. If you want to be sure nothing leaves your machine, the FAQ explains how to turn off cloud features with disable_ollama_cloud in ~/.ollama/server.json. Even locally, avoid feeding passwords, ID numbers or card details into AI tools; a model cannot keep a secret better than a locked file can.

If you connect a coding agent or other tool that lets a model read files or web pages, be aware of prompt injection, where hidden text in a document tries to give the model new instructions. Keep approval steps switched on for anything that deletes, sends or runs commands.

Next step

Install Ollama, run one small model, and spend 15 minutes on a real task with your own text, such as summarizing notes or drafting an email. Then try one Modelfile with a short system instruction and compare the results. Knowing how to write clear instructions and check answers matters more than which model you pick, and that skill carries over to every AI tool. If you want to build it step by step, Coursiv offers guided, bite-sized AI lessons you can take at your own pace. Explore Coursiv AI lessons.

Related guides: MCP tutorial and how to use Claude Code for beginners.

FAQ