Ollama Production Hardening: Security and Performance Audit

You've installed Ollama, pulled a few models, and hit the API from a handful of internal scripts. Now you need to move beyond the default setup: multi-user access, model sprawl, no resource limits, an unauthenticated HTTP API on 0.0.0.0, and no logging. Ollama security hardening requires the same discipline you apply to PostgreSQL, nginx or any other production service that handles external input and consumes significant CPU and memory. The difference is that a runaway inference job can pin every CPU core for minutes, and a malicious prompt can extract training data or abuse the model for unintended purposes.

This audit checklist walks through systemd unit hardening, API authentication, model provenance verification, resource constraints, and observability. You will edit real config files, set real limits, and add reverse-proxy layers. Each section gives you concrete directives and example values; adapt the numbers to your hardware and workload. For more on Ollama fundamentals and model selection, see Ollama Fundamentals.

Systemd Unit Hardening and Resource Limits

Ollama ships with a systemd service that runs as the ollama user but applies minimal sandboxing. Override the default unit with systemctl edit --full ollama.service and layer in security directives from systemd.exec(5). These reduce the attack surface if an attacker exploits a vulnerability in the Ollama binary or a model file parser.

Security Directives for the Ollama Service

Apply PrivateTmp=yes to isolate /tmp and /var/tmp. Set ProtectSystem=strict to make the entire filesystem read-only except for paths you explicitly allow. Use ProtectHome=yes to block access to /home and /root. Add NoNewPrivileges=yes to prevent privilege escalation via setuid binaries. Finally, grant write access only to the model storage directory with ReadWritePaths=/var/lib/ollama (adjust if you changed the default location).

[Unit]
Description=Ollama inference service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ollama
Group=ollama
ExecStart=/usr/local/bin/ollama serve
Environment="OLLAMA_HOST=127.0.0.1:11434"
Restart=on-failure
RestartSec=10s
StartLimitBurst=3

# Security
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
NoNewPrivileges=yes
ReadWritePaths=/var/lib/ollama

# Resource limits
MemoryMax=12G
CPUQuota=600%
TasksMax=4096

[Install]
WantedBy=multi-user.target

CPU and Memory Constraints

Set MemoryMax= to prevent the service from consuming all RAM. On a 16 GB system shared with other services, MemoryMax=12G leaves headroom for the OS and monitoring agents. Use CPUQuota=600% on an eight-core machine to reserve two cores for other processes (600% = six cores). TasksMax=4096 caps the number of threads and child processes; Ollama typically spawns one worker per request plus internal threads, so 4096 is generous unless you expect dozens of concurrent inference calls.

Restart and Watchdog Tuning

Configure Restart=on-failure and RestartSec=10s so systemd waits ten seconds before restarting after a crash. Set StartLimitBurst=3 to stop restarting if the service fails three times within the default 10-second window (adjust StartLimitIntervalSec= if you want a longer window). Ollama 0.4.x does not yet advertise watchdog support in its release notes, so WatchdogSec= will have no effect; check future changelogs. After editing, run systemctl daemon-reload and systemctl restart ollama. Verify with systemctl status ollama and journalctl -u ollama -f.

API Authentication and Network Binding

Ollama 0.1.x through 0.4.x binds to 0.0.0.0:11434 by default with no authentication. Any process on the network can submit prompts, pull models, or enumerate installed models. Restrict the listen address and add an authentication layer.

Binding to Localhost or a Private Interface

Set OLLAMA_HOST=127.0.0.1:11434 in the systemd unit's Environment= directive (shown above). This limits access to the local machine. If you need to serve requests from other hosts on a private network, bind to a specific interface: OLLAMA_HOST=10.0.1.5:11434. Never bind to 0.0.0.0 on a public interface without a reverse proxy in front.

Reverse Proxy with TLS and Basic Auth

For multi-host access, front Ollama with nginx or Caddy. The reverse proxy terminates TLS, enforces authentication, and rate-limits requests. Below is an nginx configuration with HTTP Basic Auth. Generate a password file with htpasswd -c /etc/nginx/.htpasswd ollama_user.

upstream ollama_backend {
    server 127.0.0.1:11434;
}

server {
    listen 443 ssl http2;
    server_name ollama.internal.example.com;

    ssl_certificate /etc/nginx/certs/ollama.crt;
    ssl_certificate_key /etc/nginx/certs/ollama.key;
    ssl_protocols TLSv1.3;

    auth_basic "Ollama API";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location / {
        proxy_pass http://ollama_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 300s;
    }
}

Test with curl -u ollama_user https://ollama.internal.example.com/api/tags. For OAuth2 or OIDC, deploy oauth2-proxy in front of nginx and configure it to validate tokens against your identity provider.

Validating Client Certificates (Optional)

In high-security environments, require mutual TLS. Configure nginx with ssl_client_certificate /etc/nginx/certs/ca.crt; and ssl_verify_client on;. Clients must present a certificate signed by your internal CA. The corresponding curl command is curl --cert client.crt --key client.key https://ollama.internal.example.com/api/generate -d '{"model":"llama3.2","prompt":"..."}'. See the nginx HTTP core module documentation for additional SSL directives.

Model Provenance and Storage Security

Models are pulled from Ollama's registry or imported from Hugging Face. As of version 0.4.x, Ollama does not verify cryptographic signatures on model files. You must establish a manual provenance process.

Verifying Model Checksums and Sources

Pull models on a bastion or CI runner, not directly on production servers. Compare the SHA256 hash of the downloaded .gguf file against the checksum published on the model card or GitHub release. The command ollama show --modelfile <model> displays the Modelfile definition but does not expose the file hash. Navigate to /var/lib/ollama/models/blobs and run sha256sum sha256-<hash> to verify the blob. If the hash does not match the upstream release, delete the model and investigate.

Restricting Model Pull Permissions

Only the ollama service user or a designated admin should write to the model directory. Prevent regular users from running ollama pull or ollama create by removing execute permissions on the Ollama binary for the other group, or by restricting access via sudo policies. For a deeper look at securing self-hosted AI services, see AI API Security.

Filesystem Permissions and Encryption at Rest

Set strict ownership and permissions on the model storage directory. Run chown -R ollama:ollama /var/lib/ollama and chmod 700 /var/lib/ollama so only the service user can read or modify model files. If you work with proprietary or regulated models, encrypt the volume with LUKS or fscrypt. Example LUKS setup: cryptsetup luksFormat /dev/sdb1, then mount it at /var/lib/ollama and configure /etc/crypttab to unlock on boot with a key file stored in the initramfs. This protects models at rest if the disk is physically removed.

Input Validation and Prompt Injection Mitigations

Ollama accepts arbitrary JSON prompts via the Ollama API; an attacker with API access can attempt prompt injection, resource exhaustion, or data exfiltration. The service itself has no built-in moderation, rate limiting, or input size caps. You must layer these controls into the reverse proxy and application code.

Rate Limiting and Request Size Caps

Configure nginx to limit requests per client IP and cap the maximum request body size. Define a rate zone in the http block and apply it to the Ollama location. Set client_max_body_size to prevent multi-gigabyte payloads that exhaust memory or disk.

http {
    limit_req_zone $binary_remote_addr zone=ollama_ratelimit:10m rate=10r/m;

    server {
        listen 443 ssl http2;
        server_name ollama.internal.example.com;
        client_max_body_size 1M;

        location / {
            limit_req zone=ollama_ratelimit burst=5 nodelay;
            proxy_pass http://127.0.0.1:11434;
            proxy_read_timeout 300s;
        }
    }
}

This configuration allows ten requests per minute per IP with a burst of five. Requests beyond that return HTTP 503. Adjust rate= and burst= based on your workload; for interactive tools, rate=30r/m may be more appropriate. The 1 MB body limit blocks attempts to smuggle large files or embed entire documents in a single prompt.

Logging and Auditing All Prompts

Enable request body logging in nginx to audit every prompt. Add a custom log format in the http block and reference it in the server. Be cautious with personally identifiable information: truncate or hash the body if needed.

http {
    log_format ollama_audit '$remote_addr - $remote_user [$time_local] '
                             '"$request" $status $body_bytes_sent '
                             '"$http_referer" "$http_user_agent" '
                             'request_body: "$request_body"';

    server {
        access_log /var/log/nginx/ollama_access.log ollama_audit;
        ...
    }
}

This writes the full request body to the log. Rotate logs daily with logrotate and retain them for at least 30 days. If you process sensitive data, replace "$request_body" with a hash: "$request_body_hash" (requires a custom nginx module or Lua script). Review logs regularly for anomalous prompts, repeated injection patterns, or attempts to exfiltrate model internals.

Prompt Filters and Content Moderation

Run user-submitted prompts through a pre-filter before sending them to Ollama. A simple Python wrapper can check for prohibited patterns or call an external moderation API. Below is an example that blocks prompts containing a hardcoded list of keywords and enforces a maximum token estimate.

import re
import requests

BLOCKLIST = [r"ignore previous instructions", r"system:", r"<script>"]
MAX_TOKENS = 2048

def moderate_prompt(prompt: str) -> bool:
    for pattern in BLOCKLIST:
        if re.search(pattern, prompt, re.IGNORECASE):
            return False
    if len(prompt.split()) > MAX_TOKENS:
        return False
    return True

def call_ollama(model: str, prompt: str):
    if not moderate_prompt(prompt):
        raise ValueError("Prompt rejected by moderation filter")
    
    response = requests.post(
        "http://127.0.0.1:11434/api/generate",
        json={"model": model, "prompt": prompt, "stream": False},
        timeout=120
    )
    response.raise_for_status()
    return response.json()

For production use, integrate a dedicated moderation service (OpenAI Moderation API, Perspective API, or an internal classifier) instead of regex. Log every rejected prompt with the reason. Remember that prompt injection is an evolving threat; update your filters as new techniques emerge. For a comprehensive approach to securing AI APIs, see AI API Security.

GPU and Compute Resource Tuning

Ollama will use all available VRAM by default. On a multi-GPU host or a shared machine, you must constrain GPU access, tune concurrency, and monitor utilization to prevent resource contention and out-of-memory errors.

CUDA Compute Limits and GPU Isolation

Set CUDA_VISIBLE_DEVICES in the systemd unit to pin Ollama to a single GPU. On a four-GPU system, assign GPU 0 to Ollama and reserve the others for training or other inference workloads.

[Service]
Environment="CUDA_VISIBLE_DEVICES=0"
Environment="OLLAMA_HOST=127.0.0.1:11434"

Verify with nvidia-smi after restarting the service. You should see Ollama processes only on GPU 0. If you need finer control, use nvidia-smi compute mode settings: nvidia-smi -c EXCLUSIVE_PROCESS on GPU 0 ensures only one process can allocate VRAM at a time. This prevents accidental oversubscription but also blocks concurrent model loads; use it only if you run a single model per GPU.

Setting OLLAMA_NUM_PARALLEL and Context Length

Tune OLLAMA_NUM_PARALLEL to control concurrent inference requests. The default is 1, meaning Ollama queues requests serially. On a 24 GB GPU running a 7B model with 4-bit quantization, you can fit four concurrent requests if you reduce the context length. Set OLLAMA_NUM_PARALLEL=4 and cap num_ctx=2048 in the model parameters (via the API or a Modelfile).

[Service]
Environment="OLLAMA_NUM_PARALLEL=4"

Test by sending four simultaneous curl requests and observing latency. If you see out-of-memory errors, reduce OLLAMA_NUM_PARALLEL or lower num_ctx. For models larger than 13B, stick with OLLAMA_NUM_PARALLEL=1 or 2 unless you have 40 GB+ of VRAM. Higher concurrency increases throughput but also increases per-request latency due to context switching and memory bandwidth saturation.

Monitoring GPU Utilization

Export GPU metrics to Prometheus with node_exporter and the NVIDIA DCGM exporter. Install DCGM exporter as a systemd service or Docker container and configure it to scrape GPU 0. A Prometheus query to detect memory saturation:

DCGM_FI_DEV_FB_USED / DCGM_FI_DEV_FB_FREE > 0.9

This fires an alert when used VRAM exceeds 90 percent of free memory. Also monitor DCGM_FI_DEV_GPU_UTIL for compute utilization and DCGM_FI_DEV_POWER_USAGE for power draw. For real-time debugging, run nvidia-smi dmon -s u to watch utilization and memory every second. If you see sustained 100 percent utilization with high request latency, either scale horizontally (add another GPU) or reduce OLLAMA_NUM_PARALLEL.

Logging, Metrics and Observability

Ollama writes all logs to stdout and stderr; systemd captures them in journald when you run the service. Structured logging, metrics collection, and request tracing are essential for diagnosing slow inference, tracking resource usage, and auditing API calls in production.

Structured Logging with journald

Set StandardOutput=journal and StandardError=journal in the systemd unit to ensure all output goes to journald. Query logs with journalctl -u ollama -f to follow in real time, or journalctl -u ollama --since "1 hour ago" for recent entries. Ollama logs include model load events, API request details, and error messages. Forward journald to a central syslog server by configuring /etc/systemd/journald.conf with ForwardToSyslog=yes and pointing SyslogFacility= to your syslog daemon. For long-term retention, export logs to a SIEM or log aggregator like Loki or Elasticsearch.

Exposing Prometheus Metrics

Ollama 0.4.x does not expose a /metrics endpoint. Instrument your reverse proxy instead. Deploy nginx-prometheus-exporter alongside nginx to export request rate, latency histograms, and error counts. Configure the exporter to scrape http://127.0.0.1:8080/stub_status (add the stub_status location to your nginx config), then add the exporter to your Prometheus scrape targets. A sample Prometheus query for 95th-percentile latency:

histogram_quantile(0.95, rate(nginx_http_request_duration_seconds_bucket[5m]))

This shows how long 95 percent of requests complete within. Set an alert if the value exceeds your SLA (for example, 10 seconds for interactive inference). Also track nginx_http_requests_total by status code to detect HTTP 503 rate-limit rejections or 500 errors from Ollama crashes.

Tracing Slow Inference Requests

Wrap Ollama API calls in OpenTelemetry spans to capture end-to-end latency. The Python example below traces a /api/generate call and logs requests slower than five seconds to syslog.

import requests
import syslog
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)
trace.get_tracer_provider().add_span_processor(
    SimpleSpanProcessor(ConsoleSpanExporter())
)

def traced_generate(model: str, prompt: str):
    with tracer.start_as_current_span("ollama.generate") as span:
        span.set_attribute("model", model)
        span.set_attribute("prompt_length", len(prompt))
        
        response = requests.post(
            "http://127.0.0.1:11434/api/generate",
            json={"model": model, "prompt": prompt, "stream": False},
            timeout=300
        )
        response.raise_for_status()
        data = response.json()
        
        duration_ns = data.get("total_duration", 0)
        duration_s = duration_ns / 1e9
        span.set_attribute("duration_seconds", duration_s)
        
        if duration_s > 5.0:
            syslog.syslog(syslog.LOG_WARNING,
                          f"Slow Ollama request: {model} took {duration_s:.2f}s")
        
        return data

Parse total_duration from the JSON response (nanoseconds) and convert to seconds. Log the model, prompt length, and duration to syslog. For distributed tracing across multiple services, export spans to Jaeger or Tempo. This gives you flame graphs showing where time is spent: model loading, prompt processing, token generation.

Backup, Disaster Recovery and Rollback Strategy

Models and configuration must be backed up separately from application data. A 7B model occupies roughly 4 GB on disk; a 70B model can exceed 40 GB. Use deduplication-aware backup tools to avoid storing identical model layers multiple times.

Automated Backups with Restic or Borg

Snapshot /var/lib/ollama daily with restic or borg. Both tools deduplicate at the chunk level, so pulling a new quantization of the same model (for example, llama3.2:7b-q4_0 versus llama3.2:7b-q8_0) reuses shared blocks. A restic backup command with a model version tag:

restic -r /mnt/backup/ollama backup /var/lib/ollama \
  --tag llama3.2:7b-20261101 \
  --exclude '*.tmp'

Schedule this in cron or a systemd timer. Retain at least seven daily snapshots and four weekly snapshots. To restore, stop Ollama, run a dry-run to verify, then restore the model directory:

systemctl stop ollama
restic -r /mnt/backup/ollama restore latest --target /var/lib/ollama --dry-run
restic -r /mnt/backup/ollama restore latest --target /var/lib/ollama
sha256sum /var/lib/ollama/models/blobs/sha256-*
systemctl start ollama
curl http://127.0.0.1:11434/api/tags

Verify checksums against your provenance records after restore. Test inference with a known prompt to confirm the model loaded correctly.

Rapid Rollback with Symlinks

Keep the previous two model versions on disk and symlink the active one. When you pull a new model, rename the old blob directory and create a symlink:

mv /var/lib/ollama/models/llama3.2-v2 /var/lib/ollama/models/llama3.2-v2-old
ollama pull llama3.2:7b
ln -sf /var/lib/ollama/models/llama3.2-v2 /var/lib/ollama/active

To roll back, delete the symlink, recreate it pointing to the old version, and restart Ollama. This takes seconds instead of waiting for a full restore from backup. Document the rollback procedure in your runbook: stop Ollama, swap symlink, start Ollama, test inference, verify logs. Run a rollback drill every quarter to ensure the team knows the steps.

Offsite Replication and Retention Policy

Replicate backups to a second site or cloud storage (S3, Backblaze B2) with restic's copy command or borg's remote repository support. Set a retention policy that balances storage cost and recovery time objective. For most deployments, keep daily backups for seven days, weekly for four weeks, and monthly for six months. Automate retention pruning with restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune. Test restores from offsite backups at least once per quarter; a backup you have never restored is not a backup.

Production Readiness Checklist

Before declaring your Ollama deployment production-ready, verify every item on this checklist. Each one represents a potential single point of failure or security gap.

1. Systemd unit hardened: PrivateTmp=yes, ProtectSystem=strict, ProtectHome=yes, NoNewPrivileges=yes, ReadWritePaths=/var/lib/ollama set.
2. API not exposed to 0.0.0.0: OLLAMA_HOST=127.0.0.1:11434 or bound to a private interface.
3. TLS and authentication in place: Reverse proxy (nginx or Caddy) terminates TLS 1.3, enforces HTTP Basic Auth or OAuth2.
4. Model directory read-only to Ollama process: chown ollama:ollama /var/lib/ollama, chmod 700, filesystem mounted with noexec,nosuid if possible.
5. Resource limits set: MemoryMax=, CPUQuota=, TasksMax= configured in the systemd unit and tested under load.
6. Logging enabled and forwarded: StandardOutput=journal, logs forwarded to central syslog or SIEM, retention policy defined.
7. Prometheus metrics from reverse proxy: nginx-prometheus-exporter or equivalent deployed, latency and error rate dashboards created.
8. GPU pinned and monitored: CUDA_VISIBLE_DEVICES set, DCGM exporter running, alerts on VRAM >90% and sustained 100% utilization.
9. Input rate limited: Nginx limit_req zone configured, client_max_body_size capped at 1 MB or less.
10. Backups tested within last 30 days: Full restore drill completed, checksums verified, rollback procedure documented.
11. Rollback procedure documented: Runbook includes symlink swap steps, restart commands, and test inference commands.
12. Security updates subscribed: GitHub watch on ollama/ollama releases enabled, internal mailing list or Slack channel for CVE notifications configured.

Print this checklist and sign off on each item with a date and operator name. Store it in your infrastructure repository alongside the systemd unit and nginx config. For a complete guide to deploying and securing self-hosted platforms, see Rocky Linux 9 + Docker + Traefik: Self-Hosted Platform.

Frequently Asked Questions