hermes-config/skills/software-development/immich-server/references/google-takeout-migration.md

# Google Takeout → Immich Migration

> Companion reference for the `immich-server` skill. Covers the end-to-end workflow: requesting the export, downloading to the server, and importing with immich-go.

## Step 1 — Request from Google

1. Go to **[takeout.google.com](https://takeout.google.com)** and sign in
2. Click **Deselect all**, then scroll down and check only **Google Photos**
3. Optional: click **All photo albums included** to select specific albums (defaults to all)
4. Scroll down, click **Next step**
5. Configure:
   - **Delivery method:** Email download link
   - **Frequency:** Export once
   - **File type:** `.zip` or `.tgz` (no preference difference for immich-go)
   - **File size:** 2GB, 10GB, or 50GB — larger = fewer parts but longer to generate
6. Click **Create export**
7. Google sends an email to the account when ready (anywhere from 30 minutes to a few days for large libraries)

## Step 2 — Transfer to Server

⚠️ **Can't download Takeout links from a headless server.** Google Takeout download URLs are session-authenticated — they require your logged-in Google browser session. `curl`/`wget` from a server hits the sign-in page every time.

**Two working approaches:**

### Option A — Download locally + SCP to server

Download each Takeout part from your **phone or computer browser** to `~/Downloads/`, then transfer over local WiFi:

```bash
# From your local machine:
scp ~/Downloads/takeout-*.zip user@192.168.x.x:~/docker/hermes/workspace/google-takeout/
```

### Option B — KasmVNC Chrome container (no local download needed)

Spin up a browser directly on the server and download the files in-place:

```bash
# On the server:
docker run -d \
  --name=chrome \
  --shm-size=2g \
  -p 6901:6901 \
  -e VNC_PW=password \
  -e LANG=en_US.UTF-8 \
  -v ~/docker/hermes/workspace/google-takeout:/home/kasm-user/Downloads \
  kasmweb/chrome:1.16.0
```

1. Open `https://192.168.x.x:6901` in your local browser
2. Login: `kasm_user` / `password` (accept self-signed cert)
3. Sign into Google inside the containerized Chrome
4. Open each Takeout link from your email — files save straight to the server
5. Done? `docker rm -f chrome`

> 🐛 **Don't use linuxserver/chromium** — it uses Selkies WebSocket which often gives a black screen. kasmweb/chrome is the proven workhorse.

#### 🛑 Pitfall: Chrome Safe Browsing kills large Takeout downloads

Large Takeout zips (30-50 GB each) trigger Chrome's **FILE_SECURITY_CHECK_FAILED** (reason code 40, danger type "UNCOMMON" / type 4). Chrome silently cancels the download, leaving orphan `.crdownload` files on disk that never finish.

**If downloads stall with .crdownload files for >1 hour, follow these steps:**

**a) Diagnose** — check Chrome's History SQLite DB to confirm Safe Browsing is the culprit:

```bash
docker cp chrome:/home/kasm-user/.config/google-chrome/Default/History /tmp/chrome_history.db
python3 -c "
import sqlite3
conn = sqlite3.connect('/tmp/chrome_history.db')
cur = conn.cursor()
cur.execute('SELECT id, target_path, state, interrupt_reason FROM downloads ORDER BY id DESC')
for r in cur.fetchall():
    print(f'ID {r[0]}: {r[1] if r[1] else \"(no path)\"} → state {r[2]} reason {r[3]}')
conn.close()
"
```

**b) Fix** — disable Safe Browsing via managed policy:

```bash
docker exec -u 0 chrome sh -c 'cat > /etc/opt/chrome/policies/managed/download_safety.json << '\''EOF'\''
{
    "DownloadRestrictions": 0,
    "SafeBrowsingEnabled": false,
    "SafeBrowsingProtectionForDownloadEnabled": false
}
EOF
'
```

**c) Restart Chrome** so policies take effect:

```bash
docker exec -u 0 chrome pkill -f chrome
# KasmVNC auto-restarts Chrome within seconds
```

**d) Clean up orphan files:**

```bash
rm -f ~/docker/hermes/workspace/google-takeout/*.crdownload
rm -f ~/docker/hermes/workspace/google-takeout/*.tmp
```

After this, reconnect to KasmVNC, re-open the Takeout download links, and retry. Chrome will no longer block them.

### Extract on the server

```bash
cd ~/docker/hermes/workspace/google-takeout
mkdir -p extracted

# Install unzip if missing
which unzip || sudo apt install -y unzip

# Extract each zip into the subdirectory (cleaner than extracting in-place)
for f in takeout-*.zip; do
  echo "Extracting $f..."
  unzip -q -o "$f" -d extracted/
done

# For .tgz files:
# for f in *.tgz; do tar -xzf "$f" -C extracted/; done

echo "Done — $(find extracted/ -type f | wc -l) files extracted"
du -sh extracted/
```

The extracted structure under `extracted/` will be a single `Takeout/` directory containing:
```
extracted/Takeout/Google Photos/
  ├── Photos from 2024/
  ├── Photos from 2025/
  ├── <album names>/
  └── ...
```

## Step 3 — Import with immich-go

```bash
# Point immich-go at the extracted directory (NOT the zip files):
/usr/local/bin/immich-go \
  --server=http://192.168.x.x:2283 \
  --api-key=YOUR_IMMICH_KEY \
  upload from-google-photos ~/docker/hermes/workspace/google-takeout/extracted/

# The tool:
# - Strips out .json metadata sidecar files automatically
# - Preserves EXIF timestamps and dates
# - Restores album/album structure from Google Takeout format
# - Skips duplicates (matched by content hash)
# - Preserves people tags and archived/trashed state

# If the API key lacks job.create permission, add --pause-immich-jobs=FALSE:
/usr/local/bin/immich-go \
  --server=http://192.168.x.x:2283 \
  --api-key=YOUR_IMMICH_KEY \
  --pause-immich-jobs=FALSE \
  upload from-google-photos ~/docker/hermes/workspace/google-takeout/extracted/
```

**⚠️ Flag gotchas:**
- Use `--server` and `--api-key` (double-dash long form). Single-dash `-server` is parsed as `-s` + `erver=` and fails.
- `upload` is a parent command — you MUST specify a subcommand: `from-google-photos` (for Takeout) or `from-folder` (for raw folders).


Progress updates scroll in terminal. For large imports (38K+ files, 184GB), expect 30-60 minutes depending on server load and disk speed.

## Step 4 — Cleanup

After confirming all photos imported successfully:

```bash
# Option A — just remove the extracted files, keep zips
rm -rf ~/docker/hermes/workspace/google-takeout/extracted

# Option B — remove everything including zips
rm -rf ~/docker/hermes/workspace/google-takeout
```

## Common Sizes

| Google Photos Library | Takeout Size Estimate |
|-----------------------|-----------------------|
| 10 GB used | ~10-12 GB |
| 50 GB used | ~50-60 GB |
| 200 GB used | ~200-240 GB |
| 2 TB used | ~2-2.5 TB |

Downloading large archives directly to the server (Option B) avoids any need to keep a desktop computer running overnight.