Ollama cheatsheet
Ollama: Server and deployment
8 common Ollama tasks.
Start the server
Start the Ollama server manually, if it isn't already running as a service.
ollama serve
Listen on all network interfaces
Allow other machines to reach the server. By default it listens only on localhost.
OLLAMA_HOST=0.0.0.0:11434 ollama serve
Caution: Ollama has no authentication. Anyone who can reach this port can use your models, so put it behind a firewall or a reverse proxy.
Store models elsewhere
Change where models are saved, for example to a larger disk.
OLLAMA_MODELS=/data/ollama/models ollama serve
Keep a model loaded longer
Keep models in memory for an hour after the last request instead of the default five minutes.
OLLAMA_KEEP_ALIVE=1h ollama serve
Run in Docker
Start Ollama in a container with a volume for models.
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Run in Docker with NVIDIA GPUs
Give the container access to all GPUs. Needs the NVIDIA Container Toolkit.
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Run a model inside the container
Run a model with the container's Ollama.
docker exec -it ollama ollama run llama3.2