hermes-config/skills/mlops/computer-vision-inference/references/immich-yolo-classifier.md

Immich YOLO Photo Classifier — Production Script

A working sequential YOLO classification pipeline for Immich photos on a self-hosted server with GTX 1050 Ti.

Script

Saved at /tmp/yolo_gpu_run.py on rayserver. Key design:

Workflow

1. Query Immich DB via docker exec immich_postgres psql — pulls all timeline IMAGE assets
2. Map virtual paths — /data/library/... → /mnt/wd-passport/immich/photos/...
3. Classify with YOLOv8n on CUDA — processes one image at a time, logs every 500
4. Move non-people, non-scenic photos to /mnt/wd-passport/immich/NO PEOPLE PHOTOS

Classification heuristic (COCO classes)

PERSON_CLASS  = 0                                → keep
NATURE_CLASSES = {16,17,18,19,20,21,22,23,24,    → keep
                  25,26,27,58,77,80}
MOVE_CLASSES   = {56,57,59–65,67,70–80,1–15}     → move (indoor/urban/vehicle/animals)
No detections + file < 400KB                      → move
No detections + file ≥ 400KB                      → keep as scenic

Running it

Script is launched with nohup so it survives terminal closure:

nohup ~/yolo_venv_cu126/bin/python3 /tmp/yolo_gpu_run.py > ~/yolo_run.log 2>&1

Check progress: tail -5 ~/yolo_run.log

Performance (GTX 1050 Ti, USB HDD)

• 16,187 images in ~40 min (6–7 img/s)
• 1,875 photos moved in a test run (12%)
• Bottleneck is disk I/O (WD Passport USB HDD), not GPU

EXIF-Aware Restore (fixes sideways missed detections — NOT always sufficient)

When YOLO misses people in rotated/portrait photos, first try PIL-based EXIF rotation. But note: EXIF-only is NOT always sufficient. Some photos are stored with rotated pixel data and no EXIF orientation flag at all. The brute-force 4-orientation approach below covers all cases.

Approach A: EXIF rotation (fast, catches flagged rotations)

from PIL import Image, ImageOps
import numpy as np
from ultralytics import YOLO

model = YOLO("yolov8n.pt")
NO_PEOPLE_DIR = "/mnt/wd-passport/immich/NO PEOPLE PHOTOS"
PHOTO_BASE = "/mnt/wd-passport/immich/photos"

for ap in files:
    # Load with EXIF rotation applied
    with Image.open(ap) as img:
        img = ImageOps.exif_transpose(img)
        if img is None:
            continue
        if img.mode != "RGB":
            img = img.convert("RGB")
        img_np = np.array(img)[:, :, ::-1]  # RGB -> BGR for YOLO

    results = model(img_np, device="cuda:0", verbose=False)

    has_person = any(
        int(box.cls[0]) == 0
        for r in results if r.boxes
        for box in r.boxes
    )

    if has_person:
        orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
        os.makedirs(os.path.dirname(orig), exist_ok=True)
        shutil.move(ap, orig)

Result from 3951 photos: 0 restores — EXIF rotation found no additional people.

✅ Approach B: 4-Orientation brute-force (definitive, catches all rotation cases)

When EXIF-only finds nothing but the user insists people exist in sideways photos, switch to trying all 4 orientations. This is the definitive approach:

import numpy as np

def try_all_orientations(ap: str, model) -> tuple[bool, str]:
    with Image.open(ap) as img:
        if img.mode != 'RGB':
            img = img.convert('RGB')
        img_np = np.array(img)[:, :, ::-1]

    orientations = {
        '0°': img_np,
        '90°': np.rot90(img_np, k=3),
        '180°': np.rot90(img_np, k=2),
        '270°': np.rot90(img_np, k=1),
    }

    for orient_name, orient_img in orientations.items():
        results = model(orient_img, device="cuda:0", verbose=False)
        for r in results:
            if r.boxes:
                for box in r.boxes:
                    if int(box.cls[0]) == 0:
                        return (True, orient_name)
    return (False, "")

# Restore loop
restored = 0
for ap in files:
    found, orientation = try_all_orientations(ap, model)
    if found:
        orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
        os.makedirs(os.path.dirname(orig), exist_ok=True)
        shutil.move(ap, orig)
        restored += 1
        print(f"RESTORED ({orientation}): {os.path.basename(ap)}")

Result from 3951 photos: 280 restored (primarily at 90° and 270°). Most missing people were in photos taken on phones that store portrait pixel data without EXIF orientation flags.

Performance: ~4.5 img/s, ~25 min for 4k photos, RAM stable at 14%.

Image Loading Edge Cases (PIL path)

When passing a numpy array (from PIL) instead of a file path to YOLO, these issues arise:

• ImageOps.exif_transpose returns None — This happens when the image has no EXIF orientation tag at all (common with screenshots, downloaded images, re-saved files). The old pattern if img is None: continue silently skips these images without counting them as errors or "still empty". Fix: reload the image fresh without EXIF if None, or fall through to 4-orientation brute force.
• RGBA PNG (4 channels): RuntimeError: expected input[1, 4, 640, 416] to have 3 channels, but got 4 channels instead. Fix: img = img.convert("RGB").
• Decompression bomb: Image size (199756800 pixels) exceeds limit of 178956970 pixels. Large panoramas/stitches (~200MP) trigger Pillow's safety limit. Fix: PIL.Image.MAX_IMAGE_PIXELS = None before loading, or skip large files.
• Corrupt JPEG: Invalid SOS parameters for sequential JPEG / Corrupt JPEG data: 1 extraneous bytes before marker. YOLO skips these cleanly in try/except.

CUDA Version Gotcha: CUDA 13.0 + Pascal GPUs

Symptom: PyTorch with CUDA 13.0 (cu130) wheels installs fine, torch.cuda.is_available() returns True, but model(image) fails with:

RuntimeError: GET was unable to find an engine to execute this computation

Root cause: CUDA 13.0 dropped support for CC 6.x compute. Memory/mgmt functions (cudaMalloc, cudaMemcpy) work, but actual compute kernels (conv2d) fail because CUBLAS 13.0 has no binary for Pascal's architecture.

Fix: Use CUDA 12.6 wheels instead (cu126). Forward-compatible with driver 580:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126

Verified on GTX 1050 Ti, driver 580.159.03.

Flattening Sorted Photos Into a Single Folder

After classification and restore, photos live under Immich's subdirectory tree (library/<uuid>/<year>/<month-day>/<filename>). To flatten into a single folder:

# Create target
mkdir -p "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos"

# Find + flatten (handles filename collisions with _1, _2 suffix)
find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type f -not -path '*/ALL Photos/*' | \
while read f; do
  base=$(basename "$f")
  dest="/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/$base"
  if [ -f "$dest" ]; then
    n=1
    while [ -f "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}" ]; do n=$((n+1)); done
    mv "$f" "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}"
  else
    mv "$f" "$dest"
  fi
done

# Clean up empty dirs
find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type d -empty -not -path '*/ALL Photos*' -not -path "/mnt/wd-passport/immich/NO PEOPLE PHOTOS" -delete

Result from 3672 files: all flattened in seconds. 1 filename collision (auto-renamed with _1 suffix).

Summary: Full Pipeline Run (16,187 Immich Photos)

Initial classification (1-orientation, GPU):   3,954 → NO PEOPLE
  ↓
EXIF-aware restore (PIL + exif_transpose):          0 restored
  ↓
4-orientation brute-force restore:                280 restored
  ↓
Final in NO PEOPLE:                             3,672 flattened into ALL Photos/

Total time: ~70 min (47 min classify + 9 min EXIF restore + 9 min 4-orientation restore + ~5 min operations)

Categorization Results (3,668 "No People" Photos)

After the full pipeline, a categorization pass was run on the 3,668 remaining photos using single-orientation YOLO to build a detection histogram:

Result: 2,743 photos had detections, 899 had no detections (blank/blurry/text-only screenshots), 26 errors (200MP panoramas).

Top categories — see the SKILL.md "Categorization by Detected Content" section for the full breakdown. The key takeaway: ~25% of sorted-out photos are blank/solid/blurry (no detections at all), and most of the detected ones are screenshots of phones/TVs/laptops or documents (books). Only ~15% are "maybe interesting" (cars, airplanes, food, clocks).

• HEIC photos — pip install pi-heif needed or they silently fail (PIL.UnidentifiedImageError: cannot identify image file)
• Disk-bound — sequential is fine, multiprocessing won't help if USB is the bottleneck
• Kill + resume — no resume state is saved. Script must restart from scratch. For prod, add a SQLite checkpoint file
• GPU vs CPU — device="cuda:0" requires PyTorch CUDA build. On Pascal GPUs (CC 6.x), PyTorch 2.x sometimes works with specific CUDA 12.x builds — test with torch.cuda.is_available() first
• "moved: X" in logs is queued, not actual — The classify phase tracks len(to_move), displayed as "moved: 2393". Actual file moves happen in a separate phase after classification completes. During classification, the destination directory stays empty.
• Immediate vs Batched file moves — Two patterns. Batched (default): collect "to_move" list during classification, move in a separate phase. Log shows "moved: X" but destination is empty until the final phase. Immediate-move: shutil.move() inside the classification loop — destination fills in real-time, log counter is actual. Ask the user which they prefer. This configuration prefers immediate moves.
• Gateway restart kills agent-launched runs — Even if launched with nohup, the process is a child of the agent's gateway session. Restarting the gateway kills it. Use cronjob(action="create", no_agent=True, script="...") for durable runs.
• Agent-launched runs waste API tokens — The agent polls progress logs mid-turn, each poll = one LLM API call. Using cronjob(no_agent=True) avoids this entirely.

Specific Error: RuntimeError: GET was unable to find an engine to execute this computation

Cause: PyTorch with CUDA 13.0 (cu130) on Pascal GPUs (CC 6.1). CUDA 13 dropped support for CC 6.x compute kernels. Memory management functions work (cudaMalloc etc.) but conv2d fails.

Fix (modern): Use CUDA 12.6 wheels instead (forward-compatible with driver 580):

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126

Fix (legacy): Python 3.10 + PyTorch 2.0.1+cu118 via uv (Python 3.10 + CUDA 11.8 build).

Specific Error: RuntimeError: Numpy is not available

Cause: PyTorch 2.0.1 was compiled against numpy 1.x; ultralytics pulls in numpy 2.x.

Fix: Pin numpy<2 after installing ultralytics.

Specific Error: RuntimeError: Error when binding input: There's no data transfer registered

Cause: ONNX Runtime's CUDAExecutionProvider fails to load because libcublasLt.so.12 is missing. The system has CUDA 13.0 installed, but onnxruntime-gpu was compiled against earlier CUDA.

Fix: Fall back to OpenVINO or use Option C (PyTorch CUDA 11.8).

Specific Error: OOM (out of memory) with multiprocessing

Symptom: Workers start, process a few hundred images, then all die simultaneously. dmesg shows OOM killer activity.

Cause: model(ap, verbose=False) without stream=True accumulates all Results objects in RAM. Each worker holds results for all images it's processed so far. With 10 workers × ~200 images each = ~2000 full-resolution feature maps retained.

Fix: Pass stream=True to prevent accumulation, or reduce workers, or switch to sequential batch processing.

Specific Warning: WARNING ⚠️ NMS time limit 2.050s exceeded

Cause: Non-maximum suppression (NMS) takes too long — typically because image decoding on a slow USB drive causes the pipeline to stall, then multiple images batch up in the worker's pipeline.

Impact: Self-correcting — NMS processes what it has and moves on. Not a hard error. Can indicate the USB drive is the bottleneck.

Fix: No action needed if results look correct. Consider reducing batch size or switching to sequential mode.