Pre-Caching Immich ML Models for GPU Acceleration

How to manually download and cache CLIP + face detection models so the ML container starts cleanly on GPU without the download→fail→clear→retry loop.

When to use this

The Immich ML container (immich-machine-learning) downloads models from HuggingFace on first startup. If the download fails or the model load fails (disk space, VRAM contention, interrupted download), it enters an infinite loop:

WARNING  Failed to load visual model 'ViT-B-32__openai'. Clearing cache.
WARNING  Failed to load detection model 'buffalo_l'. Clearing cache.
Downloading visual model 'ViT-B-32__openai' to /cache/clip/...
Downloading detection model 'buffalo_l' to /cache/facial-recognition/...
→ repeat ad infinitum

The container stays (unhealthy) and never serves ML requests.

Cache directory structure

Each model type has a fixed cache path determined by Immich's InferenceModel base class:

Model	Task (value)	Type (value)	Full Path
CLIP visual	`clip`	`visual`	`/cache/clip/ViT-B-32__openai/visual/model.onnx`
CLIP textual	`clip`	`textual`	`/cache/clip/ViT-B-32__openai/textual/model.onnx`
Face detection	`facial-recognition`	`detection`	`/cache/facial-recognition/buffalo_l/detection/model.onnx`
Face recognition	`facial-recognition`	`recognition`	`/cache/facial-recognition/buffalo_l/recognition/model.onnx`

The formula is:

settings.cache_folder / model_task.value / model_name / model_type.value / model.onnx

Where settings.cache_folder is /cache by default.

Pre-caching procedure

Run from the host against the running ML container. The venv is at /opt/venv/bin/activate and has huggingface_hub available.

1. CLIP model (smart search)

docker exec -i immich_machine_learning bash << 'SCRIPT'
source /opt/venv/bin/activate
python -c "
from huggingface_hub import snapshot_download
import os
cache_dir = '/cache/clip/ViT-B-32__openai'
os.makedirs(cache_dir, exist_ok=True)
snapshot_download(
    'immich-app/ViT-B-32__openai',
    cache_dir=cache_dir,
    local_dir=cache_dir,
    ignore_patterns=['*.armnn', '*.rknn'],
    max_workers=2,
)
print(f'CLIP model cached: {os.path.getsize(cache_dir + \"/visual/model.onnx\")/1024/1024:.0f} MB')
"
SCRIPT

2. Face detection + recognition model (buffalo_l)

docker exec -i immich_machine_learning bash << 'SCRIPT'
source /opt/venv/bin/activate
python -c "
from huggingface_hub import snapshot_download
import os
cache_dir = '/cache/facial-recognition/buffalo_l'
os.makedirs(cache_dir, exist_ok=True)
snapshot_download(
    'immich-app/buffalo_l',
    cache_dir=cache_dir,
    local_dir=cache_dir,
    ignore_patterns=['*.armnn', '*.rknn'],
    max_workers=2,
)
print(f'Detection ONNX: {os.path.getsize(cache_dir + \"/detection/model.onnx\")/1024/1024:.0f} MB')
print(f'Recognition ONNX: {os.path.getsize(cache_dir + \"/recognition/model.onnx\")/1024/1024:.0f} MB')
"
SCRIPT

3. Verify models load on GPU

docker exec -i immich_machine_learning bash << 'SCRIPT'
source /opt/venv/bin/activate
python -c "
from immich_ml.models import from_model_type
from immich_ml.schemas import ModelTask, ModelType
import time

for name, mt, task in [
    ('ViT-B-32__openai', ModelType.VISUAL, ModelTask.SEARCH),
    ('buffalo_l', ModelType.DETECTION, ModelTask.FACIAL_RECOGNITION),
    ('buffalo_l', ModelType.RECOGNITION, ModelTask.FACIAL_RECOGNITION),
]:
    m = from_model_type(name, mt, task)
    start = time.time()
    m.load()
    print(f'{name} {mt.value}: loaded in {time.time()-start:.1f}s')
"
SCRIPT

Expected output:

ViT-B-32__openai visual: loaded in 0.5s
buffalo_l detection: loaded in 0.0s
buffalo_l recognition: loaded in 1.2s

4. Restart ML container

Once models are cached, restart so the health check passes:

docker compose restart immich-machine-learning

After restart, verify the logs show clean startup with no download/load warnings:

docker logs immich_machine_learning --tail 10

5. Re-trigger ML jobs

The jobs were interrupted by the container restart. Re-trigger them:

# Login
LOGIN=$(curl -s -X POST http://localhost:2283/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"admin@example.com","password":"your-password"}')
TOK=*** -e "console.log(JSON.parse(process.argv[1]).accessToken)" -- "$LOGIN")

# Trigger each job
for JOB in smartSearch faceDetection facialRecognition thumbnailGeneration; do
  curl -s -X PUT "http://localhost:2283/api/jobs/$JOB" \
    -H "Authorization: Bearer *** \
    -H "Content-Type: application/json" \
    -d '{"command":"start"}'
done

Verifying GPU is working

After re-triggering, monitor:

# GPU memory usage — should jump from 11 MiB to 1200+ MiB
nvidia-smi --query-gpu=memory.used,utilization.gpu,temperature.gpu --format=csv,noheader

# ML container logs should show ONNX Runtime using CUDA providers
docker logs immich_machine_learning | grep -i "cuda\|provider"

# Immich server should show face detection activity
docker logs immich_server --tail 20 | grep "PersonService\|Detected"

Expected GPU profile during active processing (GTX 1050 Ti 4GB):

VRAM: ~1,200–1,400 MiB
Utilization: 1–40% (bursty, depends on queue depth)
Temp: 10–15°C above idle (e.g., 28°C → 40°C)

Performance expectations

Model	Load time	Inference	Notes
CLIP visual (335 MB)	~0.8s	~237ms	Batch size 1, output shape (1, 512)
Face detection (16 MB)	~0.0s	instant	Small model
Face recognition (166 MB)	~1.2s	varies	Depends on face count per image

Pitfalls

Container has no curl or ping. Use Python's urllib.request from the activated venv for network tests.
Model downloads require HuggingFace hub access. The immich-app/* repos are public but use snapshot_download(), not raw curl. Plain HTTPS requests return 401.
Cache dir is a Docker volume. If you recreate the container with docker compose down -v, the cache is lost and you need to pre-cache again.
Pre-caching too many models may fill VRAM. The CLIP model alone is 335 MB; buffalo_l adds ~182 MB. Total ~517 MB in VRAM, well within a 4 GB GPU. But loading ALL models simultaneously may cause OOM if other processes are using VRAM.
Model arena (MACHINE_LEARNING_MODEL_ARENA=true) can help if VRAM is tight by running models sequentially rather than keeping all in memory.

6.4 KiB Raw Permalink Blame History Unescape Escape