initial commit

2026-07-12 10:17:17 -04:00
commit dab5a4ebc6
1424 changed files with 330463 additions and 0 deletions
@@ -0,0 +1,243 @@
+# Immich YOLO Photo Classifier — Production Script
+
+A working sequential YOLO classification pipeline for Immich photos on a self-hosted server with GTX 1050 Ti.
+
+## Script
+
+Saved at `/tmp/yolo_gpu_run.py` on rayserver. Key design:
+
+### Workflow
+1. **Query Immich DB** via `docker exec immich_postgres psql` — pulls all timeline IMAGE assets
+2. **Map virtual paths** — `/data/library/...` → `/mnt/wd-passport/immich/photos/...`
+3. **Classify with YOLOv8n** on CUDA — processes one image at a time, logs every 500
+4. **Move non-people, non-scenic photos** to `/mnt/wd-passport/immich/NO PEOPLE PHOTOS`
+
+### Classification heuristic (COCO classes)
+```
+PERSON_CLASS  = 0                                → keep
+NATURE_CLASSES = {16,17,18,19,20,21,22,23,24,    → keep
+                  25,26,27,58,77,80}
+MOVE_CLASSES   = {56,57,59–65,67,70–80,1–15}     → move (indoor/urban/vehicle/animals)
+No detections + file < 400KB                      → move
+No detections + file ≥ 400KB                      → keep as scenic
+```
+
+### Running it
+Script is launched with `nohup` so it survives terminal closure:
+```bash
+nohup ~/yolo_venv_cu126/bin/python3 /tmp/yolo_gpu_run.py > ~/yolo_run.log 2>&1
+```
+
+Check progress: `tail -5 ~/yolo_run.log`
+
+### Performance (GTX 1050 Ti, USB HDD)
+- **16,187 images** in ~40 min (6–7 img/s)
+- **1,875 photos** moved in a test run (12%)
+- Bottleneck is disk I/O (WD Passport USB HDD), not GPU
+
+### EXIF-Aware Restore (fixes sideways missed detections — NOT always sufficient)
+
+When YOLO misses people in rotated/portrait photos, first try PIL-based EXIF rotation. **But note: EXIF-only is NOT always sufficient.** Some photos are stored with rotated pixel data and no EXIF orientation flag at all. The brute-force 4-orientation approach below covers all cases.
+
+#### Approach A: EXIF rotation (fast, catches flagged rotations)
+
+```python
+from PIL import Image, ImageOps
+import numpy as np
+from ultralytics import YOLO
+
+model = YOLO("yolov8n.pt")
+NO_PEOPLE_DIR = "/mnt/wd-passport/immich/NO PEOPLE PHOTOS"
+PHOTO_BASE = "/mnt/wd-passport/immich/photos"
+
+for ap in files:
+    # Load with EXIF rotation applied
+    with Image.open(ap) as img:
+        img = ImageOps.exif_transpose(img)
+        if img is None:
+            continue
+        if img.mode != "RGB":
+            img = img.convert("RGB")
+        img_np = np.array(img)[:, :, ::-1]  # RGB -> BGR for YOLO
+
+    results = model(img_np, device="cuda:0", verbose=False)
+
+    has_person = any(
+        int(box.cls[0]) == 0
+        for r in results if r.boxes
+        for box in r.boxes
+    )
+
+    if has_person:
+        orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
+        os.makedirs(os.path.dirname(orig), exist_ok=True)
+        shutil.move(ap, orig)
+```
+
+**Result from 3951 photos**: 0 restores — EXIF rotation found no additional people.
+
+#### ✅ Approach B: 4-Orientation brute-force (definitive, catches all rotation cases)
+
+When EXIF-only finds nothing but the user insists people exist in sideways photos, switch to trying all 4 orientations. This is the **definitive** approach:
+
+```python
+import numpy as np
+
+def try_all_orientations(ap: str, model) -> tuple[bool, str]:
+    with Image.open(ap) as img:
+        if img.mode != 'RGB':
+            img = img.convert('RGB')
+        img_np = np.array(img)[:, :, ::-1]
+
+    orientations = {
+        '0°': img_np,
+        '90°': np.rot90(img_np, k=3),
+        '180°': np.rot90(img_np, k=2),
+        '270°': np.rot90(img_np, k=1),
+    }
+
+    for orient_name, orient_img in orientations.items():
+        results = model(orient_img, device="cuda:0", verbose=False)
+        for r in results:
+            if r.boxes:
+                for box in r.boxes:
+                    if int(box.cls[0]) == 0:
+                        return (True, orient_name)
+    return (False, "")
+
+# Restore loop
+restored = 0
+for ap in files:
+    found, orientation = try_all_orientations(ap, model)
+    if found:
+        orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
+        os.makedirs(os.path.dirname(orig), exist_ok=True)
+        shutil.move(ap, orig)
+        restored += 1
+        print(f"RESTORED ({orientation}): {os.path.basename(ap)}")
+```
+
+**Result from 3951 photos**: **280 restored** (primarily at 90° and 270°). Most missing people were in photos taken on phones that store portrait pixel data without EXIF orientation flags.
+
+**Performance**: ~4.5 img/s, ~25 min for 4k photos, RAM stable at 14%.
+
+## Image Loading Edge Cases (PIL path)
+
+When passing a numpy array (from PIL) instead of a file path to YOLO, these issues arise:
+
+- **`ImageOps.exif_transpose` returns None** — This happens when the image has no EXIF orientation tag at all (common with screenshots, downloaded images, re-saved files). The old pattern `if img is None: continue` **silently skips these images** without counting them as errors or "still empty". Fix: reload the image fresh without EXIF if None, or fall through to 4-orientation brute force.
+- **RGBA PNG** (4 channels): `RuntimeError: expected input[1, 4, 640, 416] to have 3 channels, but got 4 channels instead`. Fix: `img = img.convert("RGB")`.
+- **Decompression bomb**: `Image size (199756800 pixels) exceeds limit of 178956970 pixels`. Large panoramas/stitches (~200MP) trigger Pillow's safety limit. Fix: `PIL.Image.MAX_IMAGE_PIXELS = None` before loading, or skip large files.
+- **Corrupt JPEG**: `Invalid SOS parameters for sequential JPEG` / `Corrupt JPEG data: 1 extraneous bytes before marker`. YOLO skips these cleanly in try/except.
+
+## CUDA Version Gotcha: CUDA 13.0 + Pascal GPUs
+
+**Symptom**: PyTorch with CUDA 13.0 (cu130) wheels installs fine, `torch.cuda.is_available()` returns True, but `model(image)` fails with:
+```
+RuntimeError: GET was unable to find an engine to execute this computation
+```
+
+**Root cause**: CUDA 13.0 dropped support for CC 6.x compute. Memory/mgmt functions (cudaMalloc, cudaMemcpy) work, but actual compute kernels (conv2d) fail because CUBLAS 13.0 has no binary for Pascal's architecture.
+
+**Fix**: Use CUDA 12.6 wheels instead (cu126). Forward-compatible with driver 580:
+```bash
+pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
+```
+Verified on GTX 1050 Ti, driver 580.159.03.
+
+## Flattening Sorted Photos Into a Single Folder
+
+After classification and restore, photos live under Immich's subdirectory tree (`library/<uuid>/<year>/<month-day>/<filename>`). To flatten into a single folder:
+
+```bash
+# Create target
+mkdir -p "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos"
+
+# Find + flatten (handles filename collisions with _1, _2 suffix)
+find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type f -not -path '*/ALL Photos/*' | \
+while read f; do
+  base=$(basename "$f")
+  dest="/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/$base"
+  if [ -f "$dest" ]; then
+    n=1
+    while [ -f "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}" ]; do n=$((n+1)); done
+    mv "$f" "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}"
+  else
+    mv "$f" "$dest"
+  fi
+done
+
+# Clean up empty dirs
+find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type d -empty -not -path '*/ALL Photos*' -not -path "/mnt/wd-passport/immich/NO PEOPLE PHOTOS" -delete
+```
+
+**Result from 3672 files**: all flattened in seconds. 1 filename collision (auto-renamed with `_1` suffix).
+
+## Summary: Full Pipeline Run (16,187 Immich Photos)
+
+```
+Initial classification (1-orientation, GPU):   3,954 → NO PEOPLE
+  ↓
+EXIF-aware restore (PIL + exif_transpose):          0 restored
+  ↓
+4-orientation brute-force restore:                280 restored
+  ↓
+Final in NO PEOPLE:                             3,672 flattened into ALL Photos/
+```
+
+**Total time**: ~70 min (47 min classify + 9 min EXIF restore + 9 min 4-orientation restore + ~5 min operations)
+
+## Categorization Results (3,668 "No People" Photos)
+
+After the full pipeline, a categorization pass was run on the 3,668 remaining photos using single-orientation YOLO to build a detection histogram:
+
+**Result**: 2,743 photos had detections, 899 had no detections (blank/blurry/text-only screenshots), 26 errors (200MP panoramas).
+
+**Top categories** — see the SKILL.md "Categorization by Detected Content" section for the full breakdown. The key takeaway: ~25% of sorted-out photos are blank/solid/blurry (no detections at all), and most of the detected ones are screenshots of phones/TVs/laptops or documents (books). Only ~15% are "maybe interesting" (cars, airplanes, food, clocks).
+- **HEIC photos** — `pip install pi-heif` needed or they silently fail (`PIL.UnidentifiedImageError: cannot identify image file`)
+- **Disk-bound** — sequential is fine, multiprocessing won't help if USB is the bottleneck
+- **Kill + resume** — no resume state is saved. Script must restart from scratch. For prod, add a SQLite checkpoint file
+- **GPU vs CPU** — `device="cuda:0"` requires PyTorch CUDA build. On Pascal GPUs (CC 6.x), PyTorch 2.x sometimes works with specific CUDA 12.x builds — test with `torch.cuda.is_available()` first
+- **"moved: X" in logs is queued, not actual** — The classify phase tracks `len(to_move)`, displayed as "moved: 2393". Actual file moves happen in a separate phase after classification completes. During classification, the destination directory stays empty.
+- **Immediate vs Batched file moves** — Two patterns. Batched (default): collect "to_move" list during classification, move in a separate phase. Log shows "moved: X" but destination is empty until the final phase. Immediate-move: `shutil.move()` inside the classification loop — destination fills in real-time, log counter is actual. Ask the user which they prefer. This configuration prefers **immediate moves**.
+- **Gateway restart kills agent-launched runs** — Even if launched with nohup, the process is a child of the agent's gateway session. Restarting the gateway kills it. Use `cronjob(action="create", no_agent=True, script="...")` for durable runs.
+- **Agent-launched runs waste API tokens** — The agent polls progress logs mid-turn, each poll = one LLM API call. Using `cronjob(no_agent=True)` avoids this entirely.
+
+### Specific Error: `RuntimeError: GET was unable to find an engine to execute this computation`
+
+**Cause**: PyTorch with CUDA 13.0 (cu130) on Pascal GPUs (CC 6.1). CUDA 13 dropped support for CC 6.x compute kernels. Memory management functions work (cudaMalloc etc.) but conv2d fails.
+
+**Fix (modern)**: Use CUDA 12.6 wheels instead (forward-compatible with driver 580):
+```bash
+pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
+```
+
+**Fix (legacy)**: Python 3.10 + PyTorch 2.0.1+cu118 via uv (Python 3.10 + CUDA 11.8 build).
+
+### Specific Error: `RuntimeError: Numpy is not available`
+
+**Cause**: PyTorch 2.0.1 was compiled against numpy 1.x; `ultralytics` pulls in numpy 2.x.
+
+**Fix**: Pin `numpy<2` after installing ultralytics.
+
+### Specific Error: `RuntimeError: Error when binding input: There's no data transfer registered`
+
+**Cause**: ONNX Runtime's CUDAExecutionProvider fails to load because `libcublasLt.so.12` is missing. The system has CUDA 13.0 installed, but onnxruntime-gpu was compiled against earlier CUDA.
+
+**Fix**: Fall back to OpenVINO or use Option C (PyTorch CUDA 11.8).
+
+### Specific Error: OOM (out of memory) with multiprocessing
+
+**Symptom**: Workers start, process a few hundred images, then all die simultaneously. `dmesg` shows OOM killer activity.
+
+**Cause**: `model(ap, verbose=False)` without `stream=True` accumulates all Results objects in RAM. Each worker holds results for all images it's processed so far. With 10 workers × ~200 images each = ~2000 full-resolution feature maps retained.
+
+**Fix**: Pass `stream=True` to prevent accumulation, or reduce workers, or switch to sequential batch processing.
+
+### Specific Warning: `WARNING ⚠️ NMS time limit 2.050s exceeded`
+
+**Cause**: Non-maximum suppression (NMS) takes too long — typically because image decoding on a slow USB drive causes the pipeline to stall, then multiple images batch up in the worker's pipeline.
+
+**Impact**: Self-correcting — NMS processes what it has and moves on. Not a hard error. Can indicate the USB drive is the bottleneck.
+
+**Fix**: No action needed if results look correct. Consider reducing batch size or switching to sequential mode.