GPU Acceleration - AgentSystems

Prerequisites

Linux with NVIDIA GPU
NVIDIA drivers and nvidia-container-toolkit installed

Configuration

After running agentsystems init, edit compose/local/docker-compose.yml and add this to the ollama service:

ollama:
  image: ollama/ollama:latest
  # ... existing configuration ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

Then run:

agentsystems up

Verify

Check GPU usage while running a model:

nvidia-smi

macOS does not support NVIDIA GPUs in Docker. AMD/Intel GPU support is not currently available.