initial commit
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
---
|
||||
name: immich-photo-classification
|
||||
description: Classify, sort, and organize Immich photos using YOLO on GPU — people detection, content categorization, and trash/keep sorting.
|
||||
trigger: |
|
||||
Load when the user asks to classify Immich photos, run YOLO on their media library,
|
||||
sort photos by content, move photos out of Immich, restore photos with missed detections,
|
||||
or organize NO PEOPLE / flagged photo folders.
|
||||
Also load when the user wants to categorize or filter their photo library by content type.
|
||||
---
|
||||
|
||||
# Immich Photo Classification Pipeline
|
||||
|
||||
Full-stack workflow for classifying Immich photos with YOLOv8n on GPU, including:
|
||||
- People/nature detection to filter non-people photos
|
||||
- EXIF-aware re-checking with multi-orientation inference
|
||||
- Content-based categorization (documents, screenshots, food, vehicles, etc.)
|
||||
- Sorting into Trash/Keep folders
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **yolov8s.pt or yolov8n.pt** — downloaded at first run (ultralytics auto-downloads)
|
||||
- **Python venv with** `ultralytics` + `pillow` + `numpy` + `torch` (CUDA)
|
||||
- **Immich server** running with PostgreSQL accessible via `docker exec`
|
||||
- **For GTX 1050 Ti (Pascal, CC 6.1) on Ubuntu 26.04:** MUST use Python 3.12 + CUDA 12.4. The system Python 3.14 ships PyTorch 2.12 with CUDA 13.0 which dropped Pascal support.
|
||||
- Install Python 3.12: `sudo add-apt-repository -y ppa:deadsnakes/ppa && sudo apt update && sudo apt install -y python3.12 python3.12-venv`
|
||||
- Create venv: `python3.12 -m venv /mnt/storage/yolo_venv_cu124`
|
||||
- Install: `pip install torch==2.5.1+cu124 torchvision==0.20.1+cu124 --index-url https://download.pytorch.org/whl/cu124 && pip install ultralytics`
|
||||
- On 4GB VRAM: use YOLOv8s (11M params). YOLOv8x (68M) will OOM.
|
||||
- **After any crash or killed run:** check `nvidia-smi` for zombie GPU processes and `kill -9` them before restarting.
|
||||
|
||||
## Step-by-Step
|
||||
|
||||
### 1. Query Immich Database
|
||||
|
||||
```python
|
||||
cmd = '''docker exec immich_postgres psql -U postgres -d immich -t -A -c "SELECT \\\"originalPath\\\" FROM asset WHERE type='IMAGE' AND visibility='timeline' AND \\\"originalPath\\\" LIKE '/data/library/%' ORDER BY \\\"originalPath\\\";" 2>/dev/null'''
|
||||
```
|
||||
|
||||
Map the DB paths to filesystem:
|
||||
```python
|
||||
PHOTO_BASE = "/mnt/wd-passport/immich/photos"
|
||||
ap = PHOTO_BASE + "/" + p[6:] # strip '/data/' prefix
|
||||
```
|
||||
|
||||
### 2. GPU Inference — CRMICAL: Fix Orientation First
|
||||
|
||||
**DO NOT pass file paths directly to YOLO for phone photos.**
|
||||
YOLO uses OpenCV (`cv2.imread()`) which does NOT read EXIF orientation flags.
|
||||
Portrait/rotated phone photos will be analyzed sideways and people will be missed.
|
||||
|
||||
**CORRECT approach — try ALL 4 orientations:**
|
||||
|
||||
```python
|
||||
from PIL import Image, ImageOps
|
||||
import numpy as np
|
||||
|
||||
with Image.open(ap) as img:
|
||||
img = ImageOps.exif_transpose(img)
|
||||
if img is None:
|
||||
img = Image.open(ap) # reload without EXIF
|
||||
if img.mode != 'RGB':
|
||||
img = img.convert('RGB') # handles RGBA, CMYK, etc.
|
||||
img_bgr = np.array(img)[:, :, ::-1] # RGB -> BGR for YOLO
|
||||
|
||||
# Try all 4 orientations
|
||||
orientations = {
|
||||
'0°': img_bgr,
|
||||
'90°': np.rot90(img_bgr, k=3),
|
||||
'180°': np.rot90(img_bgr, k=2),
|
||||
'270°': np.rot90(img_bgr, k=1),
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Move Files Immediately
|
||||
|
||||
**DO NOT batch moves at the end.** The user wants files moved as soon as they're flagged:
|
||||
|
||||
```python
|
||||
shutil.move(ap, dest) # move immediately when flagged
|
||||
```
|
||||
|
||||
### 4. Content Categorization
|
||||
|
||||
After classifying all photos, categorize by detected objects using COCO classes.
|
||||
|
||||
**Trash classes** (likely screenshots/boring):
|
||||
- `cell phone` (67) — phone screenshots
|
||||
- `tv` (62) — TV/monitor captures
|
||||
- `laptop` (63) — computer screenshots
|
||||
- `keyboard` (66), `mouse` (64), `remote` (65)
|
||||
- `toilet` (61), `chair` (56), `couch` (57), `bed` (59)
|
||||
- `refrigerator` (72), `microwave` (68), `oven` (69), `sink` (71)
|
||||
- `hair drier` (78), `toothbrush` (79), `scissors` (76)
|
||||
- No detections / blank images
|
||||
|
||||
**Keep classes** (potentially important):
|
||||
- `book` (73) — documents, receipts
|
||||
- `car` (2), `truck` (7), `airplane` (4) — vehicles/travel
|
||||
- `dining table` (60), `cup` (41), `bottle` (39) — food/meals
|
||||
- `clock` (74) — time-stamped events
|
||||
- Animals: `dog` (16), `cat` (15), `bird` (14)
|
||||
- `potted plant` (58), `vase` (75)
|
||||
- Sports: `sports ball` (32), `baseball bat` (34)
|
||||
- Everything else not in Trash set
|
||||
|
||||
Logic: if ANY detected class is a Keep class → Keep folder.
|
||||
If ALL detected classes are Trash classes OR no detections → Trash folder.
|
||||
|
||||
### 5. Sort into Flat Folders
|
||||
|
||||
Use a flat structure (no subdirectories) for the sorted output:
|
||||
|
||||
```python
|
||||
ALL_PHOTOS → Trash/ and Keep/
|
||||
```
|
||||
|
||||
Handle filename collisions with numbered suffixes (`_1`, `_2`, etc.).
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **RGBA images** (PNGs with alpha): convert to RGB before passing to YOLO, otherwise: `expected input[1, 4, 640, 416] to have 3 channels`
|
||||
- **Decompression bomb protection**: Pillow rejects images >178M pixels by default. These are typically 200MP phone panorama stitches. Log and skip them.
|
||||
- **RAM accumulation**: `model(source)` without `stream=True` accumulates results. For single-image inference in a loop this is manageable (results go out of scope), but for large batches consider `stream=True`.
|
||||
- **GPU compatibility**: GTX 1050 Ti (Pascal, CC 6.1) needs PyTorch with CUDA 12.x. CUDA 13.0+ PyTorch may fail. Use `/home/ray/yolo_venv_cu124/bin/python` (Python 3.12 + PyTorch 2.5.1+cu124) on rayserver.
|
||||
- **`ImageOps.exif_transpose()` can return None**: Always handle this case — reload the image without EXIF fallback.
|
||||
- **File naming collisions**: When flattening subdirectories, use numbered suffixes to avoid overwriting.
|
||||
- **Empty directories**: After moving all files out of subdirectories, clean them up with `find ... -type d -empty -delete`.
|
||||
@@ -0,0 +1,243 @@
|
||||
# Immich YOLO Photo Classifier — Production Script
|
||||
|
||||
A working sequential YOLO classification pipeline for Immich photos on a self-hosted server with GTX 1050 Ti.
|
||||
|
||||
## Script
|
||||
|
||||
Saved at `/tmp/yolo_gpu_run.py` on rayserver. Key design:
|
||||
|
||||
### Workflow
|
||||
1. **Query Immich DB** via `docker exec immich_postgres psql` — pulls all timeline IMAGE assets
|
||||
2. **Map virtual paths** — `/data/library/...` → `/mnt/wd-passport/immich/photos/...`
|
||||
3. **Classify with YOLOv8n** on CUDA — processes one image at a time, logs every 500
|
||||
4. **Move non-people, non-scenic photos** to `/mnt/wd-passport/immich/NO PEOPLE PHOTOS`
|
||||
|
||||
### Classification heuristic (COCO classes)
|
||||
```
|
||||
PERSON_CLASS = 0 → keep
|
||||
NATURE_CLASSES = {16,17,18,19,20,21,22,23,24, → keep
|
||||
25,26,27,58,77,80}
|
||||
MOVE_CLASSES = {56,57,59–65,67,70–80,1–15} → move (indoor/urban/vehicle/animals)
|
||||
No detections + file < 400KB → move
|
||||
No detections + file ≥ 400KB → keep as scenic
|
||||
```
|
||||
|
||||
### Running it
|
||||
Script is launched with `nohup` so it survives terminal closure:
|
||||
```bash
|
||||
nohup ~/yolo_venv_cu126/bin/python3 /tmp/yolo_gpu_run.py > ~/yolo_run.log 2>&1
|
||||
```
|
||||
|
||||
Check progress: `tail -5 ~/yolo_run.log`
|
||||
|
||||
### Performance (GTX 1050 Ti, USB HDD)
|
||||
- **16,187 images** in ~40 min (6–7 img/s)
|
||||
- **1,875 photos** moved in a test run (12%)
|
||||
- Bottleneck is disk I/O (WD Passport USB HDD), not GPU
|
||||
|
||||
### EXIF-Aware Restore (fixes sideways missed detections — NOT always sufficient)
|
||||
|
||||
When YOLO misses people in rotated/portrait photos, first try PIL-based EXIF rotation. **But note: EXIF-only is NOT always sufficient.** Some photos are stored with rotated pixel data and no EXIF orientation flag at all. The brute-force 4-orientation approach below covers all cases.
|
||||
|
||||
#### Approach A: EXIF rotation (fast, catches flagged rotations)
|
||||
|
||||
```python
|
||||
from PIL import Image, ImageOps
|
||||
import numpy as np
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
NO_PEOPLE_DIR = "/mnt/wd-passport/immich/NO PEOPLE PHOTOS"
|
||||
PHOTO_BASE = "/mnt/wd-passport/immich/photos"
|
||||
|
||||
for ap in files:
|
||||
# Load with EXIF rotation applied
|
||||
with Image.open(ap) as img:
|
||||
img = ImageOps.exif_transpose(img)
|
||||
if img is None:
|
||||
continue
|
||||
if img.mode != "RGB":
|
||||
img = img.convert("RGB")
|
||||
img_np = np.array(img)[:, :, ::-1] # RGB -> BGR for YOLO
|
||||
|
||||
results = model(img_np, device="cuda:0", verbose=False)
|
||||
|
||||
has_person = any(
|
||||
int(box.cls[0]) == 0
|
||||
for r in results if r.boxes
|
||||
for box in r.boxes
|
||||
)
|
||||
|
||||
if has_person:
|
||||
orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
|
||||
os.makedirs(os.path.dirname(orig), exist_ok=True)
|
||||
shutil.move(ap, orig)
|
||||
```
|
||||
|
||||
**Result from 3951 photos**: 0 restores — EXIF rotation found no additional people.
|
||||
|
||||
#### ✅ Approach B: 4-Orientation brute-force (definitive, catches all rotation cases)
|
||||
|
||||
When EXIF-only finds nothing but the user insists people exist in sideways photos, switch to trying all 4 orientations. This is the **definitive** approach:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
|
||||
def try_all_orientations(ap: str, model) -> tuple[bool, str]:
|
||||
with Image.open(ap) as img:
|
||||
if img.mode != 'RGB':
|
||||
img = img.convert('RGB')
|
||||
img_np = np.array(img)[:, :, ::-1]
|
||||
|
||||
orientations = {
|
||||
'0°': img_np,
|
||||
'90°': np.rot90(img_np, k=3),
|
||||
'180°': np.rot90(img_np, k=2),
|
||||
'270°': np.rot90(img_np, k=1),
|
||||
}
|
||||
|
||||
for orient_name, orient_img in orientations.items():
|
||||
results = model(orient_img, device="cuda:0", verbose=False)
|
||||
for r in results:
|
||||
if r.boxes:
|
||||
for box in r.boxes:
|
||||
if int(box.cls[0]) == 0:
|
||||
return (True, orient_name)
|
||||
return (False, "")
|
||||
|
||||
# Restore loop
|
||||
restored = 0
|
||||
for ap in files:
|
||||
found, orientation = try_all_orientations(ap, model)
|
||||
if found:
|
||||
orig = ap.replace(NO_PEOPLE_DIR, PHOTO_BASE)
|
||||
os.makedirs(os.path.dirname(orig), exist_ok=True)
|
||||
shutil.move(ap, orig)
|
||||
restored += 1
|
||||
print(f"RESTORED ({orientation}): {os.path.basename(ap)}")
|
||||
```
|
||||
|
||||
**Result from 3951 photos**: **280 restored** (primarily at 90° and 270°). Most missing people were in photos taken on phones that store portrait pixel data without EXIF orientation flags.
|
||||
|
||||
**Performance**: ~4.5 img/s, ~25 min for 4k photos, RAM stable at 14%.
|
||||
|
||||
## Image Loading Edge Cases (PIL path)
|
||||
|
||||
When passing a numpy array (from PIL) instead of a file path to YOLO, these issues arise:
|
||||
|
||||
- **`ImageOps.exif_transpose` returns None** — This happens when the image has no EXIF orientation tag at all (common with screenshots, downloaded images, re-saved files). The old pattern `if img is None: continue` **silently skips these images** without counting them as errors or "still empty". Fix: reload the image fresh without EXIF if None, or fall through to 4-orientation brute force.
|
||||
- **RGBA PNG** (4 channels): `RuntimeError: expected input[1, 4, 640, 416] to have 3 channels, but got 4 channels instead`. Fix: `img = img.convert("RGB")`.
|
||||
- **Decompression bomb**: `Image size (199756800 pixels) exceeds limit of 178956970 pixels`. Large panoramas/stitches (~200MP) trigger Pillow's safety limit. Fix: `PIL.Image.MAX_IMAGE_PIXELS = None` before loading, or skip large files.
|
||||
- **Corrupt JPEG**: `Invalid SOS parameters for sequential JPEG` / `Corrupt JPEG data: 1 extraneous bytes before marker`. YOLO skips these cleanly in try/except.
|
||||
|
||||
## CUDA Version Gotcha: CUDA 13.0 + Pascal GPUs
|
||||
|
||||
**Symptom**: PyTorch with CUDA 13.0 (cu130) wheels installs fine, `torch.cuda.is_available()` returns True, but `model(image)` fails with:
|
||||
```
|
||||
RuntimeError: GET was unable to find an engine to execute this computation
|
||||
```
|
||||
|
||||
**Root cause**: CUDA 13.0 dropped support for CC 6.x compute. Memory/mgmt functions (cudaMalloc, cudaMemcpy) work, but actual compute kernels (conv2d) fail because CUBLAS 13.0 has no binary for Pascal's architecture.
|
||||
|
||||
**Fix**: Use CUDA 12.6 wheels instead (cu126). Forward-compatible with driver 580:
|
||||
```bash
|
||||
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
|
||||
```
|
||||
Verified on GTX 1050 Ti, driver 580.159.03.
|
||||
|
||||
## Flattening Sorted Photos Into a Single Folder
|
||||
|
||||
After classification and restore, photos live under Immich's subdirectory tree (`library/<uuid>/<year>/<month-day>/<filename>`). To flatten into a single folder:
|
||||
|
||||
```bash
|
||||
# Create target
|
||||
mkdir -p "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos"
|
||||
|
||||
# Find + flatten (handles filename collisions with _1, _2 suffix)
|
||||
find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type f -not -path '*/ALL Photos/*' | \
|
||||
while read f; do
|
||||
base=$(basename "$f")
|
||||
dest="/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/$base"
|
||||
if [ -f "$dest" ]; then
|
||||
n=1
|
||||
while [ -f "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}" ]; do n=$((n+1)); done
|
||||
mv "$f" "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/ALL Photos/${base%.*}_$n.${base##*.}"
|
||||
else
|
||||
mv "$f" "$dest"
|
||||
fi
|
||||
done
|
||||
|
||||
# Clean up empty dirs
|
||||
find "/mnt/wd-passport/immich/NO PEOPLE PHOTOS/" -type d -empty -not -path '*/ALL Photos*' -not -path "/mnt/wd-passport/immich/NO PEOPLE PHOTOS" -delete
|
||||
```
|
||||
|
||||
**Result from 3672 files**: all flattened in seconds. 1 filename collision (auto-renamed with `_1` suffix).
|
||||
|
||||
## Summary: Full Pipeline Run (16,187 Immich Photos)
|
||||
|
||||
```
|
||||
Initial classification (1-orientation, GPU): 3,954 → NO PEOPLE
|
||||
↓
|
||||
EXIF-aware restore (PIL + exif_transpose): 0 restored
|
||||
↓
|
||||
4-orientation brute-force restore: 280 restored
|
||||
↓
|
||||
Final in NO PEOPLE: 3,672 flattened into ALL Photos/
|
||||
```
|
||||
|
||||
**Total time**: ~70 min (47 min classify + 9 min EXIF restore + 9 min 4-orientation restore + ~5 min operations)
|
||||
|
||||
## Categorization Results (3,668 "No People" Photos)
|
||||
|
||||
After the full pipeline, a categorization pass was run on the 3,668 remaining photos using single-orientation YOLO to build a detection histogram:
|
||||
|
||||
**Result**: 2,743 photos had detections, 899 had no detections (blank/blurry/text-only screenshots), 26 errors (200MP panoramas).
|
||||
|
||||
**Top categories** — see the SKILL.md "Categorization by Detected Content" section for the full breakdown. The key takeaway: ~25% of sorted-out photos are blank/solid/blurry (no detections at all), and most of the detected ones are screenshots of phones/TVs/laptops or documents (books). Only ~15% are "maybe interesting" (cars, airplanes, food, clocks).
|
||||
- **HEIC photos** — `pip install pi-heif` needed or they silently fail (`PIL.UnidentifiedImageError: cannot identify image file`)
|
||||
- **Disk-bound** — sequential is fine, multiprocessing won't help if USB is the bottleneck
|
||||
- **Kill + resume** — no resume state is saved. Script must restart from scratch. For prod, add a SQLite checkpoint file
|
||||
- **GPU vs CPU** — `device="cuda:0"` requires PyTorch CUDA build. On Pascal GPUs (CC 6.x), PyTorch 2.x sometimes works with specific CUDA 12.x builds — test with `torch.cuda.is_available()` first
|
||||
- **"moved: X" in logs is queued, not actual** — The classify phase tracks `len(to_move)`, displayed as "moved: 2393". Actual file moves happen in a separate phase after classification completes. During classification, the destination directory stays empty.
|
||||
- **Immediate vs Batched file moves** — Two patterns. Batched (default): collect "to_move" list during classification, move in a separate phase. Log shows "moved: X" but destination is empty until the final phase. Immediate-move: `shutil.move()` inside the classification loop — destination fills in real-time, log counter is actual. Ask the user which they prefer. This configuration prefers **immediate moves**.
|
||||
- **Gateway restart kills agent-launched runs** — Even if launched with nohup, the process is a child of the agent's gateway session. Restarting the gateway kills it. Use `cronjob(action="create", no_agent=True, script="...")` for durable runs.
|
||||
- **Agent-launched runs waste API tokens** — The agent polls progress logs mid-turn, each poll = one LLM API call. Using `cronjob(no_agent=True)` avoids this entirely.
|
||||
|
||||
### Specific Error: `RuntimeError: GET was unable to find an engine to execute this computation`
|
||||
|
||||
**Cause**: PyTorch with CUDA 13.0 (cu130) on Pascal GPUs (CC 6.1). CUDA 13 dropped support for CC 6.x compute kernels. Memory management functions work (cudaMalloc etc.) but conv2d fails.
|
||||
|
||||
**Fix (modern)**: Use CUDA 12.6 wheels instead (forward-compatible with driver 580):
|
||||
```bash
|
||||
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
|
||||
```
|
||||
|
||||
**Fix (legacy)**: Python 3.10 + PyTorch 2.0.1+cu118 via uv (Python 3.10 + CUDA 11.8 build).
|
||||
|
||||
### Specific Error: `RuntimeError: Numpy is not available`
|
||||
|
||||
**Cause**: PyTorch 2.0.1 was compiled against numpy 1.x; `ultralytics` pulls in numpy 2.x.
|
||||
|
||||
**Fix**: Pin `numpy<2` after installing ultralytics.
|
||||
|
||||
### Specific Error: `RuntimeError: Error when binding input: There's no data transfer registered`
|
||||
|
||||
**Cause**: ONNX Runtime's CUDAExecutionProvider fails to load because `libcublasLt.so.12` is missing. The system has CUDA 13.0 installed, but onnxruntime-gpu was compiled against earlier CUDA.
|
||||
|
||||
**Fix**: Fall back to OpenVINO or use Option C (PyTorch CUDA 11.8).
|
||||
|
||||
### Specific Error: OOM (out of memory) with multiprocessing
|
||||
|
||||
**Symptom**: Workers start, process a few hundred images, then all die simultaneously. `dmesg` shows OOM killer activity.
|
||||
|
||||
**Cause**: `model(ap, verbose=False)` without `stream=True` accumulates all Results objects in RAM. Each worker holds results for all images it's processed so far. With 10 workers × ~200 images each = ~2000 full-resolution feature maps retained.
|
||||
|
||||
**Fix**: Pass `stream=True` to prevent accumulation, or reduce workers, or switch to sequential batch processing.
|
||||
|
||||
### Specific Warning: `WARNING ⚠️ NMS time limit 2.050s exceeded`
|
||||
|
||||
**Cause**: Non-maximum suppression (NMS) takes too long — typically because image decoding on a slow USB drive causes the pipeline to stall, then multiple images batch up in the worker's pipeline.
|
||||
|
||||
**Impact**: Self-correcting — NMS processes what it has and moves on. Not a hard error. Can indicate the USB drive is the bottleneck.
|
||||
|
||||
**Fix**: No action needed if results look correct. Consider reducing batch size or switching to sequential mode.
|
||||
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Immich YOLO Photo Classifier Pipeline
|
||||
Classifies photos as: has-people, scenic, or neither.
|
||||
Moves non-people/non-scenic photos to NO PEOPLE PHOTOS folder.
|
||||
|
||||
Usage:
|
||||
/mnt/storage/yolo_venv/bin/python3 this_script.py
|
||||
|
||||
Requirements:
|
||||
- /mnt/storage/yolo_venv/ with ultralytics installed
|
||||
- ~/yolov8n_openvino_model/ (exported from yolov8n.pt)
|
||||
- Immich PostgreSQL container running
|
||||
- Photos on /mnt/wd-passport/immich/photos/
|
||||
"""
|
||||
import multiprocessing, os, time, warnings, subprocess, shutil
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
PHOTO_BASE = "/mnt/wd-passport/immich/photos"
|
||||
NO_PEOPLE_DIR = "/mnt/wd-passport/immich/NO PEOPLE PHOTOS"
|
||||
DB_QUERY = '''docker exec immich_postgres psql -U postgres -d immich -t -A -c "SELECT \\\"originalPath\\\" FROM asset WHERE type='IMAGE' AND visibility='timeline' AND \\\"originalPath\\\" LIKE '/data/library/%' ORDER BY \\\"originalPath\\\";" 2>/dev/null'''
|
||||
|
||||
PERSON_CLASS = 0
|
||||
NATURE_CLASSES = {16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 58, 77, 80}
|
||||
MOVE_CLASSES = {56, 57, 59, 60, 61, 62, 63, 64, 65, 67, 70, 71, 72, 73, 74, 75, 76, 78, 79,
|
||||
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
|
||||
|
||||
def worker(args):
|
||||
model_path, paths = args
|
||||
from ultralytics import YOLO
|
||||
import warnings
|
||||
warnings.filterwarnings("ignore")
|
||||
model = YOLO(model_path, task="detect")
|
||||
results = []
|
||||
for ap in paths:
|
||||
try:
|
||||
r = model(ap, verbose=False)
|
||||
has_person = has_nature = has_move = False
|
||||
for r2 in r:
|
||||
if r2.boxes:
|
||||
for box in r2.boxes:
|
||||
cls = int(box.cls[0])
|
||||
if cls == PERSON_CLASS: has_person = True
|
||||
elif cls in NATURE_CLASSES: has_nature = True
|
||||
elif cls in MOVE_CLASSES: has_move = True
|
||||
if has_person: results.append((ap, "keep_people"))
|
||||
elif has_nature: results.append((ap, "keep_scenic"))
|
||||
elif has_move: results.append((ap, "move"))
|
||||
else: results.append((ap, "check_size"))
|
||||
except Exception as e:
|
||||
results.append((ap, f"error:{e}"))
|
||||
return results
|
||||
|
||||
if __name__ == "__main__":
|
||||
multiprocessing.set_start_method("forkserver", force=True)
|
||||
num_workers = min(multiprocessing.cpu_count(), 8)
|
||||
os.makedirs(NO_PEOPLE_DIR, exist_ok=True)
|
||||
model_path = os.path.expanduser("~/yolov8n_openvino_model/")
|
||||
|
||||
print(f"[1] Querying DB...", flush=True)
|
||||
result = subprocess.run(DB_QUERY, shell=True, capture_output=True, text=True, timeout=120)
|
||||
db_paths = [p.strip() for p in result.stdout.strip().split('\n') if p.strip()]
|
||||
|
||||
print(f"[2] Mapping {len(db_paths)} paths...", flush=True)
|
||||
actual = []
|
||||
for p in db_paths:
|
||||
ap = PHOTO_BASE + "/" + p[6:]
|
||||
if os.path.exists(ap):
|
||||
actual.append(ap)
|
||||
print(f" {len(actual)} accessible files", flush=True)
|
||||
|
||||
print(f"[3] Classifying with {num_workers} workers...", flush=True)
|
||||
chunks = [actual[i::num_workers] for i in range(num_workers)]
|
||||
chunk_args = [(model_path, c) for c in chunks]
|
||||
t_start = time.time()
|
||||
with multiprocessing.Pool(num_workers) as pool:
|
||||
all_results = pool.map(worker, chunk_args)
|
||||
elapsed = time.time() - t_start
|
||||
|
||||
keep_people = keep_scenic = errors = 0
|
||||
to_move, check_size = [], []
|
||||
for chunk in all_results:
|
||||
for ap, d in chunk:
|
||||
if d == "keep_people": keep_people += 1
|
||||
elif d == "keep_scenic": keep_scenic += 1
|
||||
elif d == "move": to_move.append(ap)
|
||||
elif d == "check_size": check_size.append(ap)
|
||||
else: errors += 1
|
||||
|
||||
print(f"\n YOLO: {elapsed/60:.1f}min ({len(actual)/elapsed:.1f} img/s)", flush=True)
|
||||
print(f" People:{keep_people} Scenic:{keep_scenic} Move:{len(to_move)} Check:{len(check_size)} Err:{errors}", flush=True)
|
||||
|
||||
print(f"[4] Size check on {len(check_size)} undetected...", flush=True)
|
||||
for ap in check_size:
|
||||
try:
|
||||
if os.path.getsize(ap) < 400000:
|
||||
to_move.append(ap)
|
||||
except: pass
|
||||
print(f" Now {len(to_move)} to move", flush=True)
|
||||
|
||||
print(f"[5] Moving {len(to_move)} photos...", flush=True)
|
||||
moved = file_errors = 0
|
||||
for ap in to_move:
|
||||
try:
|
||||
rel = os.path.relpath(ap, PHOTO_BASE)
|
||||
dest = os.path.join(NO_PEOPLE_DIR, rel)
|
||||
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||
shutil.move(ap, dest)
|
||||
moved += 1
|
||||
except Exception as e:
|
||||
file_errors += 1
|
||||
|
||||
print(f"\nDONE! Moved {moved} photos", flush=True)
|
||||
print(f" Total:{len(actual)} Kept(people):{keep_people} Kept(scenic):{keep_scenic} Moved:{moved} Errors:{file_errors}", flush=True)
|
||||
print(f" Time: {(time.time()-t_start)/60:.1f}min", flush=True)
|
||||
print(f" Output: {NO_PEOPLE_DIR}", flush=True)
|
||||
@@ -0,0 +1,34 @@
|
||||
# Video Content Detection + Renaming — Production Script
|
||||
|
||||
A working YOLO-based pipeline for detecting video content from frames and renaming generically-named recovered video files (FILE000.MOV pattern).
|
||||
|
||||
## Script
|
||||
|
||||
Saved at `/tmp/rename_videos.py` and `/tmp/rename_videos2.py` on rayserver.
|
||||
|
||||
### Workflow
|
||||
1. List all files in the target directory starting with "FILE" (generically-named recovered files)
|
||||
2. Extract a video frame at 3 seconds using ffmpeg; fall back to 1 second for short clips
|
||||
3. Run YOLOv8n on CPU for content detection
|
||||
4. Build descriptive filename: `{people_count}_people_{object1}_{object2}.mov`
|
||||
5. Handle filename collisions with `_1`, `_2` suffixes
|
||||
6. Files with no detections → `scene.mov`
|
||||
|
||||
### Key design decisions
|
||||
- **CPU inference** — Viable for 80-100 files at ~3-5s/file. GPU not needed for this scale
|
||||
- **ffmpeg single-frame extraction** — Much faster than processing the entire video
|
||||
- **3-second offset** — Early enough to avoid blank intros, late enough to have content
|
||||
- **startswith("FILE") guard** — Makes the script idempotent (won't rename already-named files)
|
||||
|
||||
### Performance reference
|
||||
- **80 files**: ~4-6 minutes total (i7-10700K, CPU)
|
||||
- **~60 files** get descriptive names, **~20** remain as `scene` (blank/dark)
|
||||
|
||||
### Full script (80-file version)
|
||||
See `/tmp/rename_videos.py` and `/tmp/rename_videos2.py` for the two-phase version that handled 80 files.
|
||||
|
||||
### Edge cases
|
||||
- `.3GP` files are valid video containers — include them
|
||||
- Short clips (<3s): ffmpeg clips to available duration, produces a valid frame
|
||||
- Broken headers: ffmpeg hangs if file is truncated — always set `timeout=20`
|
||||
- Collisions: use `while os.path.exists(dst)` loop with incrementing counter
|
||||
Reference in New Issue
Block a user