hermes-config/skills/gaming/game-streaming/references/troubleshooting-crashes.md

# Troubleshooting Sunshine Crashes

Diagnostic workflow for when Sunshine is offline, won't start, or crashes.

## Quick debug sequence

```bash
# 1. Is it running?
systemctl --user status sunshine --no-pager

# 2. Any recent crashes in systemd journal?
journalctl --user -u sunshine --no-pager -n 50

# 3. Check Sunshine's own log (⚠ overwritten on restart — copy first!)
cat ~/.config/sunshine/sunshine.log | grep -i "fatal\|error\|hang\|terminate"

# 4. Stale processes?
ps aux | grep -i sunshine | grep -v grep

# 5. Port conflicts?
ss -tlnp | grep -E "47984|47989|47990|48010"

# 6. Restart
systemctl --user restart sunshine
```

## Common crash patterns

### "Hang detected! Session failed to terminate in 10 seconds"

**Symptom:** Sunshine dies after a Moonlight client connects or disconnects. Log shows:
```
Fatal: Hang detected! Session failed to terminate in 10 seconds.
```

**Cause:** NVENC encoder or X11 capture pipeline hung during session teardown. The internal 10-second watchdog fires and kills the process.

**Common root cause — missing window manager:** Sunshine's X11 capture requires a running window manager or desktop environment on DISPLAY=:0. An X server alone (login screen, headless with no WM) allows capture to start but teardown of the X11 pipeline hangs without a WM managing the window resources. This produces a reproducible Hang detected crash every time a Moonlight client attempts a session.

**Diagnostic — check WM before restarting:**
```bash
# Is a window manager running on :0?
ps aux | grep -iE "openbox|xfce|gnome|kde|mutter|xfwm" | grep -v grep

# Is X11 alive?
DISPLAY=:0 xdpyinfo 2>&1 | head -5

# If X11 is running but no WM, start one:
DISPLAY=:0 openbox --replace &
# Then restart Sunshine
systemctl --user restart sunshine
```

**Fix:** If a WM is running, restart Sunshine — this may be a transient encoder hang. If it happens repeatedly and a WM is present:
- Check for GPU driver instability (`nvidia-smi` for errors)
- Stop co-resident LLM models to free VRAM before gaming (`ollama stop`)
- Consider increasing session timeout (Sunshine Web UI → Configuration → General)
- If on a headless server, ensure the dummy plug hasn't become unseated — no connected display causes similar hang behavior

### Silent death (log just stops, no error)

**Symptom:** Sunshine.log has startup messages then just ends. No Fatal, no Error, no "Terminate handler called."

**Possible causes:**
- OOM kill (check `dmesg | grep -i "out of memory"`)
- SIGKILL from systemd (check `journalctl --user -u sunshine` for signal)
- Second "Hang detected" crash where the log was overwritten before the Fatal line was flushed to disk

**Fix:** Restart and watch live: `tail -f ~/.config/sunshine/sunshine.log` in another terminal, then connect a Moonlight client and disconnect. Observe the teardown sequence.

### "Couldn't bind RTSP server to port [48010], Address already in use"

**Symptom:** Logged as `Fatal` during startup but Sunshine continues running.

**Cause:** A stale Flatpak wrapper process from a previous crash still holds port 48010. 

**Impact:** Moonlight uses RTSP (port 48010) for streaming protocol negotiation. Without it, clients get "RTSP handshake failed" (error 10060/connection timeout) even though Sunshine appears running and the HTTPS ports (47984, 47989) and Web UI (47990) are up.

**Fix:** 
```bash
# Find and kill the stale wrapper (not the new /usr/bin/sunshine)
ps aux | grep sunshine | grep -v grep
kill <stale-pid>
# Then restart so Sunshine rebinds all ports cleanly
systemctl --user restart sunshine
# Verify all four ports are listening
ss -tlnp | grep -E "47984|47989|47990|48010"
```

### Clean SIGTERM ("Terminate handler called")

**Symptom:** Log shows `Info: Terminate handler called` with no preceding error.

**Cause:** Something sent SIGTERM — manual stop (`systemctl --user stop`), systemd cleanup, or a Hermes agent session issuing restart.

**Fix:** Just restart. Not a crash — check system journal around the timestamp for what sent the signal.

### RTSP connection reset / "No pending session" (error 10054)

**Symptom:** Moonlight client reports "Starting RTSP handshake failed: error 10054" (Windows) or "Connection reset by peer." Sunshine log shows:
```
Debug: No pending session for incoming RTSP connection
```
Sunshine is running, all ports are listening (verified with `ss -tlnp | grep sunshine`), and HTTPS/HTTP serverinfo requests succeed — but every RTSP connection is immediately reset.

**Cause:** The Moonlight client was never paired, or its pairing was lost. Sunshine's RTSP server requires a pending session created by the HTTP pairing flow BEFORE accepting RTSP connections. With zero paired clients, there is never a pending session, so every RTSP connection gets RST immediately.

**Diagnostic — check pairing state:**
```bash
python3 -c "
import json
with open('$HOME/.config/sunshine/sunshine_state.json') as f:
    state = json.load(f)
paired = state.get('paired_clients', [])
print(f'Paired clients: {len(paired)}')
"
```
If the count is 0, the client must be paired.

**Fix — pair the client:**
1. Open Moonlight, click "+", enter `192.168.50.98` (or let it auto-discover)
2. Moonlight prompts for a PIN
3. Open Sunshine Web UI at `https://<server-ip>:47990/pin`
4. Enter the PIN shown in Moonlight into the Web UI
5. After pairing, RTSP connections will be accepted

**Troubleshooting IP Mismatch during pairing:** If the Web UI says "Success!" but Moonlight still says "PC Offline" and `sunshine_state.json` still shows 0 paired clients, this is an **IP routing mismatch**. Sunshine drops the pairing if the client IP Moonlight is using (e.g., Tailscale `100.x.x.x`) doesn't match the IP you used in the browser to access the Web UI (e.g., LAN `192.168.x.x`). **Fix:** Make sure the IP address typed into Moonlight's "Add PC" dialog *exactly matches* the IP address in your browser's address bar when entering the PIN.

**Note on `origin_pin_allowed`:** With `origin_pin_allowed = pc` in sunshine.conf, only localhost connections auto-pair. All LAN/WAN clients must enter a PIN. If `origin_pin_allowed = lan`, Tailscale clients get "Request Timed out" during pairing because Sunshine rejects the non-LAN source address.

**Error code reference:**
| Error | Meaning | Likely cause |
|---|---|---|
| 10060 | Connection timeout | Port not listening, firewall, Sunshine not running, stale process holding port 48010, OR **Sunshine crashed instantly upon connection (check log for audio `pa_simple_new` errors)** |
| 10054 | Connection reset by peer | RTSP server rejecting — unpaired client, or RTSP server module broken |

### Instant Crash on Connection (Audio Init Failure)

**Symptom:** Moonlight connects successfully via HTTP, logs show `New streaming session started`, but Sunshine crashes immediately. Moonlight reports `Error 10060` (Timeout / PC Offline) because the server died before the RTSP streaming ports could be reached.

**Log shows:**
```
Error: pa_simple_new() failed: Invalid argument
Error: Unable to initialize audio capture. The stream will not have audio.
```

**Cause:** In Sunshine v2026+, hardcoded `audio_sink = auto_null` or `virtual_sink = auto_null` in `sunshine.conf` causes a fatal crash during PulseAudio/PipeWire initialization when the client attempts to start the stream.

**Fix:** Remove `audio_sink` and `virtual_sink` from `~/.config/sunshine/sunshine.conf` and restart Sunshine. Allow Sunshine to auto-detect the PipeWire virtual sinks.

## Log locations

| Source | Path | Persistence |
|---|---|---|
| Sunshine app log | `~/.config/sunshine/sunshine.log` | **Overwritten on restart** — copy before restarting |
| Systemd journal (user) | `journalctl --user -u sunshine` | Persistent across restarts |
| System journal | `journalctl --since "HH:MM" --until "HH:MM"` | Persistent, broader system context |
| Sunshine state | `~/.config/sunshine/sunshine_state.json` | Persists across restarts (client pairings, app list) |

## Preserving crash logs

Sunshine.log is the most useful for debugging but is wiped on restart. Always copy first:

```bash
cp ~/.config/sunshine/sunshine.log ~/sunshine-crash-$(date +%Y%m%d-%H%M).log
```

Then restart and compare the old crash log against the new startup log.