hermes-config/skills/mlops/computer-vision-inference/references/immich-classification-pipeline.md

---
name: immich-photo-classification
description: Classify, sort, and organize Immich photos using YOLO on GPU — people detection, content categorization, and trash/keep sorting.
trigger: |
  Load when the user asks to classify Immich photos, run YOLO on their media library,
  sort photos by content, move photos out of Immich, restore photos with missed detections,
  or organize NO PEOPLE / flagged photo folders.
  Also load when the user wants to categorize or filter their photo library by content type.
---

# Immich Photo Classification Pipeline

Full-stack workflow for classifying Immich photos with YOLOv8n on GPU, including:
- People/nature detection to filter non-people photos
- EXIF-aware re-checking with multi-orientation inference
- Content-based categorization (documents, screenshots, food, vehicles, etc.)
- Sorting into Trash/Keep folders

## Prerequisites

- **yolov8s.pt or yolov8n.pt** — downloaded at first run (ultralytics auto-downloads)
- **Python venv with** `ultralytics` + `pillow` + `numpy` + `torch` (CUDA)
- **Immich server** running with PostgreSQL accessible via `docker exec`
- **For GTX 1050 Ti (Pascal, CC 6.1) on Ubuntu 26.04:** MUST use Python 3.12 + CUDA 12.4. The system Python 3.14 ships PyTorch 2.12 with CUDA 13.0 which dropped Pascal support.
  - Install Python 3.12: `sudo add-apt-repository -y ppa:deadsnakes/ppa && sudo apt update && sudo apt install -y python3.12 python3.12-venv`
  - Create venv: `python3.12 -m venv /mnt/storage/yolo_venv_cu124`
  - Install: `pip install torch==2.5.1+cu124 torchvision==0.20.1+cu124 --index-url https://download.pytorch.org/whl/cu124 && pip install ultralytics`
  - On 4GB VRAM: use YOLOv8s (11M params). YOLOv8x (68M) will OOM.
- **After any crash or killed run:** check `nvidia-smi` for zombie GPU processes and `kill -9` them before restarting.

## Step-by-Step

### 1. Query Immich Database

```python
cmd = '''docker exec immich_postgres psql -U postgres -d immich -t -A -c "SELECT \\\"originalPath\\\" FROM asset WHERE type='IMAGE' AND visibility='timeline' AND \\\"originalPath\\\" LIKE '/data/library/%' ORDER BY \\\"originalPath\\\";" 2>/dev/null'''
```

Map the DB paths to filesystem:
```python
PHOTO_BASE = "/mnt/wd-passport/immich/photos"
ap = PHOTO_BASE + "/" + p[6:]  # strip '/data/' prefix
```

### 2. GPU Inference — CRMICAL: Fix Orientation First

**DO NOT pass file paths directly to YOLO for phone photos.**  
YOLO uses OpenCV (`cv2.imread()`) which does NOT read EXIF orientation flags.  
Portrait/rotated phone photos will be analyzed sideways and people will be missed.

**CORRECT approach — try ALL 4 orientations:**

```python
from PIL import Image, ImageOps
import numpy as np

with Image.open(ap) as img:
    img = ImageOps.exif_transpose(img)
    if img is None:
        img = Image.open(ap)  # reload without EXIF
    if img.mode != 'RGB':
        img = img.convert('RGB')  # handles RGBA, CMYK, etc.
    img_bgr = np.array(img)[:, :, ::-1]  # RGB -> BGR for YOLO

# Try all 4 orientations
orientations = {
    '0°': img_bgr,
    '90°': np.rot90(img_bgr, k=3),
    '180°': np.rot90(img_bgr, k=2),
    '270°': np.rot90(img_bgr, k=1),
}
```

### 3. Move Files Immediately

**DO NOT batch moves at the end.** The user wants files moved as soon as they're flagged:

```python
shutil.move(ap, dest)  # move immediately when flagged
```

### 4. Content Categorization

After classifying all photos, categorize by detected objects using COCO classes.

**Trash classes** (likely screenshots/boring):
- `cell phone` (67) — phone screenshots
- `tv` (62) — TV/monitor captures
- `laptop` (63) — computer screenshots
- `keyboard` (66), `mouse` (64), `remote` (65)
- `toilet` (61), `chair` (56), `couch` (57), `bed` (59)
- `refrigerator` (72), `microwave` (68), `oven` (69), `sink` (71)
- `hair drier` (78), `toothbrush` (79), `scissors` (76)
- No detections / blank images

**Keep classes** (potentially important):
- `book` (73) — documents, receipts
- `car` (2), `truck` (7), `airplane` (4) — vehicles/travel
- `dining table` (60), `cup` (41), `bottle` (39) — food/meals
- `clock` (74) — time-stamped events
- Animals: `dog` (16), `cat` (15), `bird` (14)
- `potted plant` (58), `vase` (75)
- Sports: `sports ball` (32), `baseball bat` (34)
- Everything else not in Trash set

Logic: if ANY detected class is a Keep class → Keep folder.  
If ALL detected classes are Trash classes OR no detections → Trash folder.

### 5. Sort into Flat Folders

Use a flat structure (no subdirectories) for the sorted output:

```python
ALL_PHOTOS → Trash/ and Keep/
```

Handle filename collisions with numbered suffixes (`_1`, `_2`, etc.).

## Pitfalls

- **RGBA images** (PNGs with alpha): convert to RGB before passing to YOLO, otherwise: `expected input[1, 4, 640, 416] to have 3 channels`
- **Decompression bomb protection**: Pillow rejects images >178M pixels by default. These are typically 200MP phone panorama stitches. Log and skip them.
- **RAM accumulation**: `model(source)` without `stream=True` accumulates results. For single-image inference in a loop this is manageable (results go out of scope), but for large batches consider `stream=True`.
- **GPU compatibility**: GTX 1050 Ti (Pascal, CC 6.1) needs PyTorch with CUDA 12.x. CUDA 13.0+ PyTorch may fail. Use `/home/ray/yolo_venv_cu124/bin/python` (Python 3.12 + PyTorch 2.5.1+cu124) on rayserver.
- **`ImageOps.exif_transpose()` can return None**: Always handle this case — reload the image without EXIF fallback.
- **File naming collisions**: When flattening subdirectories, use numbered suffixes to avoid overwriting.
- **Empty directories**: After moving all files out of subdirectories, clean them up with `find ... -type d -empty -delete`.