initial commit
This commit is contained in:
@@ -0,0 +1,501 @@
|
||||
---
|
||||
name: vps-reverse-proxy
|
||||
description: "Route all self-hosted services through a cheap VPS reverse proxy via Tailscale — zero ports open on home router, SSL on the VPS, services stay at home."
|
||||
version: 1.1.0
|
||||
author: ray
|
||||
platforms: [linux]
|
||||
created_by: "agent"
|
||||
category: devops
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [vps, reverse-proxy, tailscale, nginx, ssl, duckdns, self-hosting]
|
||||
---
|
||||
|
||||
# VPS Reverse Proxy for Self-Hosted Services
|
||||
|
||||
Route all self-hosted services (Immich, Paperless, Home Assistant, Mealie, etc.) through a cheap cloud VPS. The VPS acts as the public entry point — nginx with Let's Encrypt SSL — and tunnels traffic to the home server via Tailscale. Home router has zero ports open to the internet.
|
||||
|
||||
## When to use
|
||||
|
||||
- User has multiple self-hosted services (10+ nginx sites) exposed via port forwarding on their home router
|
||||
- User wants to hide their home IP, reduce attack surface, and absorb DDoS at the VPS
|
||||
- User already has Tailscale connecting home server and wants to extend it
|
||||
- User has a cheap VPS ($4-6/mo) with 4GB+ RAM, 30GB+ disk, 1TB+ bandwidth
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Internet → VPS (ports 80 + 443 only, nginx SSL)
|
||||
↳ Tailscale WireGuard tunnel (41641 UDP)
|
||||
↳ Home server (zero ports open to internet)
|
||||
↳ Immich:2283, Paperless:8010, HA:8123, Mealie:3449, etc.
|
||||
```
|
||||
|
||||
All service data stays on the home server. The VPS only runs nginx + certbot + Tailscale. Bandwidth: a VPS with 32TB/mo handles Immich photo browsing (23K assets) and all other services without issue.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- VPS with Ubuntu (OVH, Contabo, Hetzner, DigitalOcean — any provider)
|
||||
- Root or sudo SSH access to the VPS
|
||||
- Tailscale account with home server already on the tailnet
|
||||
- DuckDNS domain (or any domain with API-updatable DNS)
|
||||
|
||||
## Step 1: Provision VPS
|
||||
|
||||
Order the cheapest instance with 4GB+ RAM. OVH VPS Starter or Contabo VPS S are ~$5/mo. Note the public IPv4 address.
|
||||
|
||||
## Step 2: Initial VPS setup
|
||||
|
||||
```bash
|
||||
# SSH in as provided user (ubuntu on OVH cloud images)
|
||||
ssh ubuntu@<vps-ip>
|
||||
|
||||
# Install essentials
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y nginx certbot python3-certbot-nginx curl ufw
|
||||
|
||||
# Firewall — only 22, 80, 443
|
||||
sudo ufw allow 22/tcp
|
||||
sudo ufw allow 80/tcp
|
||||
sudo ufw allow 443/tcp
|
||||
sudo ufw --force enable
|
||||
```
|
||||
|
||||
## Step 3: Tailscale
|
||||
|
||||
```bash
|
||||
curl -fsSL https://tailscale.com/install.sh | sudo sh
|
||||
sudo tailscale up --accept-routes --accept-dns=false
|
||||
```
|
||||
|
||||
This prints an auth URL. Open it in a browser already logged into Tailscale. After auth, verify the home server is reachable:
|
||||
|
||||
```bash
|
||||
tailscale status
|
||||
# Should show: rayserver 100.93.253.36 linux -
|
||||
|
||||
curl http://100.93.253.36:2283/api/server/ping
|
||||
# {"res":"pong"}
|
||||
```
|
||||
|
||||
### Tailscale flags
|
||||
|
||||
- `--accept-routes`: lets VPS reach subnets advertised by other tailnet nodes
|
||||
- `--accept-dns=false`: prevent Tailscale from overwriting `/etc/resolv.conf` (keep VPS DNS for Let's Encrypt challenges)
|
||||
|
||||
## Step 4: DNS
|
||||
|
||||
### Option A: Porkbun domain (recommended for >5 services)
|
||||
|
||||
DuckDNS caps at 5 entries. **IMPORTANT:** The DuckDNS update cron job must target the VPS IP explicitly, not auto-detect (which records the home IP). See `references/duckdns-vps-ip.md`.. For 6+ services, use a proper domain (~$11/yr at Porkbun). Porkbun supports wildcard DNS — one `*.graj-media.com` A record covers all subdomains.
|
||||
|
||||
**Porkbun API setup** — requires three things:
|
||||
1. API key from Porkbun → Account → API Access
|
||||
2. Secret API key from the same page
|
||||
3. Domain API access enabled: Porkbun → Account Settings → "Allow API access for all domains"
|
||||
|
||||
**DNS creation via API** — first delete any parking records, then add A records:
|
||||
|
||||
```bash
|
||||
# Delete parking/default records (CNAME/ALIAS pointing to uixie.porkbun.com)
|
||||
curl -sk -X POST "https://api.porkbun.com/api/json/v3/dns/retrieve/graj-media.com" \
|
||||
-d '{"apikey":"pk1_...","secretapikey":"sk1_..."}' | jq '.records[] | select(.type=="CNAME" or .type=="ALIAS") | .id'
|
||||
|
||||
# Delete each
|
||||
curl -sk -X POST "https://api.porkbun.com/api/json/v3/dns/delete/graj-media.com/<id>" \
|
||||
-d '{"apikey":"pk1_...","secretapikey":"sk1_..."}'
|
||||
|
||||
# Add wildcard A record
|
||||
curl -sk -X POST "https://api.porkbun.com/api/json/v3/dns/create/graj-media.com" \
|
||||
-d '{"apikey":"pk1_...","secretapikey":"sk1_...","name":"*","type":"A","content":"51.81.84.34","ttl":300}'
|
||||
|
||||
# Add root A record
|
||||
curl -sk -X POST "https://api.porkbun.com/api/json/v3/dns/create/graj-media.com" \
|
||||
-d '{"apikey":"pk1_...","secretapikey":"sk1_...","name":"","type":"A","content":"51.81.84.34","ttl":300}'
|
||||
```
|
||||
|
||||
**Pitfall:** Porkbun returns `"conflict"` errors if parking/default CNAME/ALIAS records exist. Delete them first, then add A records.
|
||||
|
||||
**Pitfall:** If the API returns `"Domain is not opted in to API access"`, go to Porkbun → Account Settings and enable the global API access toggle (separate from the API key generation page).
|
||||
|
||||
### Option B: DuckDNS (free, max 5 entries)
|
||||
|
||||
DuckDNS does NOT support dotted subdomains (`immich.grajmedia.duckdns.org`). Use the **hyphen format** instead: `immich-grajmedia.duckdns.org`.
|
||||
|
||||
On [duckdns.org](https://duckdns.org), add all domains comma-separated in the domains field:
|
||||
|
||||
```
|
||||
grajmedia,immich-grajmedia,paperless-grajmedia,ha-grajmedia,shopproquote-grajmedia,mealie-grajmedia,audiobookshelf-grajmedia
|
||||
```
|
||||
|
||||
Set the IP to the VPS public IP. Update. Verify:
|
||||
|
||||
```bash
|
||||
dig +short immich-grajmedia.duckdns.org @1.1.1.1
|
||||
# Should return VPS IP
|
||||
```
|
||||
|
||||
## Step 5: nginx reverse proxy configs
|
||||
|
||||
Each service gets its own nginx server block with its own DuckDNS subdomain. Port mappings from Tailscale IP to home server:
|
||||
|
||||
| Service | Home server port | VPS domain |
|
||||
|---------|-----------------|------------|
|
||||
| Immich | 2283 (direct) | immich-grajmedia.duckdns.org |
|
||||
| Paperless | 8010 (direct) | paperless-grajmedia.duckdns.org |
|
||||
| Home Assistant | 8123 (direct) | ha-grajmedia.duckdns.org |
|
||||
| Mealie | 3449 (home nginx SSL) | mealie-grajmedia.duckdns.org |
|
||||
| Audiobookshelf | 13378 (direct) | audiobookshelf-grajmedia.duckdns.org |
|
||||
| ShopProQuote | 443 (home nginx SSL) | shopproquote-grajmedia.duckdns.org |
|
||||
|
||||
### Template for direct-HTTP services
|
||||
|
||||
Most Docker services expose HTTP on a port (Immich, Paperless, HA, Audiobookshelf):
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name <service>-grajmedia.duckdns.org;
|
||||
client_max_body_size 500M;
|
||||
|
||||
location / {
|
||||
proxy_pass http://100.93.253.36:<port>;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_read_timeout 86400;
|
||||
proxy_buffering off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template for home-nginx-SSL services (DuckDNS)
|
||||
|
||||
Some services sit behind the home server's nginx with SSL only (Mealie on 3449, ShopProQuote on 443). The VPS must proxy via HTTPS. For DuckDNS subdomains (`<service>-grajmedia.duckdns.org`), the home server nginx only knows the DuckDNS name, so the VPS must hardcode the Host header and SNI name:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name <service>-grajmedia.duckdns.org;
|
||||
client_max_body_size 500M;
|
||||
|
||||
location / {
|
||||
proxy_pass https://100.93.253.36:<ssl_port>;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host grajmedia.duckdns.org;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_ssl_verify off;
|
||||
proxy_ssl_server_name on;
|
||||
proxy_ssl_name grajmedia.duckdns.org;
|
||||
proxy_read_timeout 86400;
|
||||
proxy_buffering off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The critical directives for DuckDNS-style HTTPS proxying to home nginx:
|
||||
- `proxy_ssl_verify off` — home nginx uses Let's Encrypt for `grajmedia.duckdns.org`, not the Tailscale IP
|
||||
- `proxy_ssl_server_name on` + `proxy_ssl_name grajmedia.duckdns.org` — SNI hostname the home nginx expects
|
||||
- `proxy_set_header Host grajmedia.duckdns.org` — hardcoded to match the home nginx server_name (DuckDNS subdomains are NOT known to home nginx; only the base DuckDNS domain is)
|
||||
|
||||
### Template for home-nginx-SSL services (custom domain)
|
||||
|
||||
When using a custom domain like `graj-media.com` with per-subdomain Let's Encrypt certs on the VPS, forward the original Host header. Each subdomain (`mealie.graj-media.com`, `vault.graj-media.com`, `plex.graj-media.com`) must ALSO be added as a `server_name` on the home server's nginx config:
|
||||
|
||||
**VPS nginx:**
|
||||
|
||||
```nginx
|
||||
server {
|
||||
server_name <service>.graj-media.com;
|
||||
client_max_body_size 100M;
|
||||
|
||||
location / {
|
||||
proxy_pass https://100.93.253.36:<home_port>;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host; # forward original subdomain
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_ssl_verify off;
|
||||
proxy_ssl_server_name on;
|
||||
proxy_ssl_name grajmedia.duckdns.org; # home nginx cert SAN
|
||||
proxy_read_timeout 86400;
|
||||
proxy_buffering off;
|
||||
}
|
||||
|
||||
listen 443 ssl; # managed by Certbot
|
||||
ssl_certificate /etc/letsencrypt/live/<service>.graj-media.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/<service>.graj-media.com/privkey.pem;
|
||||
include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||
}
|
||||
```
|
||||
|
||||
**Home server nginx — must include the subdomain in server_name:**
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen <port> ssl;
|
||||
server_name grajmedia.duckdns.org <service>.graj-media.com; # ← CRITICAL
|
||||
# ... rest of the config
|
||||
}
|
||||
```
|
||||
|
||||
Key differences from DuckDNS template:
|
||||
- `proxy_set_header Host $host;` — forward the original subdomain, NOT a hardcoded value. The home nginx matches it via `server_name`.
|
||||
- Each subdomain gets its own Let's Encrypt cert on the VPS (certbot `-d <service>.graj-media.com`)
|
||||
- The home server's `server_name` must include `<service>.graj-media.com` — always verify this when adding a new subdomain.
|
||||
|
||||
Enable each site and test:
|
||||
|
||||
```bash
|
||||
sudo ln -sf /etc/nginx/sites-available/<service> /etc/nginx/sites-enabled/
|
||||
sudo rm -f /etc/nginx/sites-enabled/default
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
Test each before DNS propagates:
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}' -H 'Host: <service>-grajmedia.duckdns.org' http://localhost/
|
||||
```
|
||||
|
||||
## Step 6: Let's Encrypt SSL
|
||||
|
||||
After DNS propagates for all subdomains, provision SSL for all at once:
|
||||
|
||||
```bash
|
||||
sudo certbot --nginx --non-interactive --agree-tos --email <email> \
|
||||
-d grajmedia.duckdns.org \
|
||||
-d immich-grajmedia.duckdns.org \
|
||||
-d paperless-grajmedia.duckdns.org \
|
||||
-d ha-grajmedia.duckdns.org \
|
||||
-d shopproquote-grajmedia.duckdns.org \
|
||||
-d mealie-grajmedia.duckdns.org \
|
||||
-d audiobookshelf-grajmedia.duckdns.org
|
||||
```
|
||||
|
||||
Certbot auto-renews. It edits the nginx configs to add `listen 443 ssl` and the cert paths.
|
||||
|
||||
### Adding a new subdomain to an existing multi-domain cert
|
||||
|
||||
When you already have a cert covering N subdomains and need to add one more (e.g. adding `gitea.graj-media.com` to an existing `graj-media.com` cert), use `--expand`:
|
||||
|
||||
```bash
|
||||
# 1. Create the nginx server block on the VPS first (certbot needs it to exist)
|
||||
ssh ubuntu@<vps-ip> 'sudo tee /etc/nginx/sites-available/<new-service> > /dev/null << '\''EOF'\''
|
||||
server {
|
||||
server_name <new-service>.graj-media.com;
|
||||
client_max_body_size 512M;
|
||||
|
||||
location / {
|
||||
proxy_pass https://100.93.253.36:<home_port>;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_ssl_verify off;
|
||||
proxy_read_timeout 86400;
|
||||
proxy_buffering off;
|
||||
}
|
||||
|
||||
listen 443 ssl;
|
||||
ssl_certificate /etc/letsencrypt/live/graj-media.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/graj-media.com/privkey.pem;
|
||||
include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||
}
|
||||
EOF
|
||||
sudo ln -sf /etc/nginx/sites-available/<new-service> /etc/nginx/sites-enabled/'
|
||||
|
||||
# 2. Expand the existing cert — list ALL existing domains PLUS the new one
|
||||
ssh ubuntu@<vps-ip> 'sudo certbot certonly --nginx --non-interactive --agree-tos \
|
||||
--email <email> \
|
||||
--cert-name graj-media.com \
|
||||
-d graj-media.com \
|
||||
-d existing1.graj-media.com \
|
||||
-d existing2.graj-media.com \
|
||||
-d existing3.graj-media.com \
|
||||
-d <new-service>.graj-media.com'
|
||||
|
||||
# 3. Reload nginx on the VPS
|
||||
ssh ubuntu@<vps-ip> 'sudo systemctl reload nginx'
|
||||
|
||||
# 4. Add the new subdomain to the HOME server's nginx server_name
|
||||
# (on the home server, edit the service's nginx config)
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
**Pitfall:** The `--expand` flag is required when adding domains to an existing cert, NOT `--force-renewal` or bare `certonly`. Without `--expand`, certbot creates a separate cert file instead of extending the existing one. The nginx `ssl_certificate` paths remain the same (certbot updates the live symlink), so no config changes needed.
|
||||
|
||||
**Pitfall:** List ALL existing domains every time. Omitting one drops it from the renewed cert. Use `openssl x509 -in /etc/letsencrypt/live/<cert-name>/fullchain.pem -noout -text | grep DNS:` to see the current SANs before expanding.
|
||||
|
||||
**Pitfall:** The VPS nginx server block for the new subdomain MUST exist before running certbot — certbot uses the nginx plugin to verify the domain. Create the config first (it can point to a non-existent backend for a moment), run certbot (which adds the SSL directives), then ensure the proxy config is correct.
|
||||
|
||||
**Pitfall:** For custom domains (graj-media.com), the VPS cert is a multi-domain cert (NOT a wildcard). Each new subdomain requires an explicit certbot expand. DNS wildcard records (`*.graj-media.com`) handle routing but NOT certificate coverage — certbot must still list every subdomain in the `-d` flags.
|
||||
|
||||
## Step 8: ShopProQuote LLM proxy
|
||||
|
||||
If ShopProQuote calls `/llm/` to a local Ollama on the home server, add a location block to the ShopProQuote nginx config on the VPS:
|
||||
|
||||
```nginx
|
||||
location /llm/ {
|
||||
proxy_pass http://100.93.253.36:11434/;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_read_timeout 120;
|
||||
}
|
||||
```
|
||||
|
||||
## Step 9: Static sites (no Docker backend)
|
||||
|
||||
For static HTML/CSS/JS sites served directly by nginx (no Docker container), see `references/static-site-deployment.md` for the complete pattern including home nginx `root` + `try_files`, file permissions, VPS HTTPS proxy, and certbot per-subdomain SSL.
|
||||
|
||||
## Step 10: Landing page
|
||||
|
||||
Create a simple HTML landing page at `grajmedia.duckdns.org` linking to all services:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name grajmedia.duckdns.org;
|
||||
|
||||
location = / {
|
||||
return 200 '<!DOCTYPE html>
|
||||
<html><head><title>grajmedia</title>
|
||||
<style>body{font-family:-apple-system,sans-serif;max-width:600px;margin:60px auto;padding:20px;background:#111;color:#eee}
|
||||
a{color:#4fc3f7;text-decoration:none;display:block;padding:8px 0;font-size:18px}
|
||||
a:hover{color:#81d4fa}h1{font-size:28px;margin-bottom:24px}</style></head>
|
||||
<body><h1>grajmedia services</h1>
|
||||
<a href=https://immich-grajmedia.duckdns.org>Immich — photos</a>
|
||||
<a href=https://paperless-grajmedia.duckdns.org>Paperless — documents</a>
|
||||
<a href=https://ha-grajmedia.duckdns.org>Home Assistant</a>
|
||||
<a href=https://mealie-grajmedia.duckdns.org>Mealie — recipes</a>
|
||||
<a href=https://audiobookshelf-grajmedia.duckdns.org>Audiobookshelf</a>
|
||||
<a href=https://shopproquote-grajmedia.duckdns.org>ShopProQuote</a>
|
||||
</body></html>';
|
||||
add_header Content-Type text/html;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Updating the landing page
|
||||
|
||||
The page is rendered via `return 200 '...'` in nginx. To add or remove services, edit `/etc/nginx/sites-available/grajmedia` and reload nginx.
|
||||
|
||||
## Post-migration: clean up home server
|
||||
|
||||
After verifying all services are accessible through the VPS:
|
||||
|
||||
1. **Remove port forwards** on the ASUS router — no more ports 443, 3443-3450 exposed
|
||||
2. **Optionally remove SSL from home nginx** — services are now accessed via VPS SSL, so home nginx can drop to HTTP-only (simpler config, faster local access)
|
||||
3. **Keep Tailscale running** — it's the backbone of the tunnel
|
||||
|
||||
## Security hardening the VPS
|
||||
|
||||
The VPS is the public face of all services. Apply these hardening steps to any service exposed through the VPS.
|
||||
|
||||
### Hide nginx version
|
||||
|
||||
```bash
|
||||
sudo sed -i 's/server_tokens build;/server_tokens off;/' /etc/nginx/nginx.conf
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### Add rate limiting
|
||||
|
||||
Add a zone to the http block in `/etc/nginx/nginx.conf`:
|
||||
```nginx
|
||||
limit_req_zone $binary_remote_addr zone=<zone_name>:10m rate=5r/m;
|
||||
```
|
||||
|
||||
Apply per-service in each server block:
|
||||
```nginx
|
||||
limit_req zone=<zone_name> burst=5 nodelay;
|
||||
```
|
||||
|
||||
### Security headers
|
||||
|
||||
Add to the location block of each VPS nginx server block. Headers at the server block level may be dropped when a location block with `proxy_pass` exists — put them inside `location`:
|
||||
```nginx
|
||||
location / {
|
||||
# ... proxy_pass, proxy_set_header directives ...
|
||||
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-Frame-Options "SAMEORIGIN" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=(), midi=(), sync-xhr=(), clipboard-read=(), clipboard-write=()" always;
|
||||
}
|
||||
```
|
||||
|
||||
### Install fail2ban for nginx HTTP auth
|
||||
|
||||
The default nginx-http-auth filter watches **error.log** (not access.log):
|
||||
```bash
|
||||
sudo apt-get install -y fail2ban
|
||||
sudo tee /etc/fail2ban/jail.d/nginx-http-auth.conf << 'EOF'
|
||||
[nginx-http-auth]
|
||||
enabled = true
|
||||
port = http,https
|
||||
filter = nginx-http-auth
|
||||
logpath = /var/log/nginx/error.log
|
||||
maxretry = 5
|
||||
bantime = 3600
|
||||
findtime = 300
|
||||
EOF
|
||||
sudo systemctl restart fail2ban
|
||||
```
|
||||
|
||||
Verify: `sudo fail2ban-client status nginx-http-auth`
|
||||
|
||||
### Pitfall: `sites-enabled` is a copy, not a symlink
|
||||
|
||||
When a VPS nginx site config is created as a **separate file** in `sites-enabled/` (rather than a symlink to `sites-available/`), future edits to `sites-available/` are silently ignored. nginx continues reading the stale copy.
|
||||
|
||||
```bash
|
||||
# Check
|
||||
ls -la /etc/nginx/sites-enabled/<service>
|
||||
# "-rw-r--r--" = copy (bad), "lrwxrwxrwx" = symlink (good)
|
||||
|
||||
# Fix
|
||||
sudo rm /etc/nginx/sites-enabled/<service>
|
||||
sudo ln -s /etc/nginx/sites-available/<service> /etc/nginx/sites-enabled/<service>
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
This affects the `sudo ln -sf` command in the enable step — if the site was originally created by a process that writes files directly (e.g., certbot, manual copy, migration from another host), it may be a standalone file.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Subdomain server_name mismatch (custom domain)**: When adding a new subdomain on the VPS (SSL cert + nginx server block), you MUST also add that subdomain to the HOME server nginx server_name for the target service. Without this, the home nginx routes traffic to the default server block (404 or wrong content). Fix: update the home nginx server_name to include <subdomain>.graj-media.com, then reload nginx. Verify with `curl -sI -H 'Host: <subdomain>.graj-media.com' http://localhost:<home_port>`.
|
||||
- **VPS Host header uses bare apex domain instead of $host or subdomain**: When hand-writing a VPS nginx config for a custom-domain subdomain, it's easy to write `proxy_set_header Host graj-media.com;` (just the root domain) instead of `proxy_set_header Host $host;` (forwards the original subdomain verbatim). This drops the subdomain prefix, so the home nginx never matches the server_name. **Symptoms**: the request hits the home nginx default server block (often a 404 or wrong service) instead of the intended one. **Fix**: use `proxy_set_header Host $host;` on the VPS (for custom domain with per-subdomain certs) or hardcode the base DuckDNS domain (for the DuckDNS pattern). **Diagnostic**: SSH into the home server and run `curl -sI -H 'Host: mealie.graj-media.com' http://localhost:9925/` — if the service responds correctly locally, the problem is the Host header sent by the VPS. <br><br>**Three-way decision for Host header on VPS**:<br>1. `proxy_set_header Host $host;` — use when the VPS has its own per-subdomain cert AND the home server's server_name includes that subdomain. This preserves the original hostname through the chain.<br>2. `proxy_set_header Host grajmedia.duckdns.org;` — use when the VPS uses a DuckDNS subdomain and the home server's server_name is only the base DuckDNS domain. The home nginx doesn't know about the hyphenated DuckDNS subdomain, so you hardcode the one name it recognizes.<br>3. `proxy_set_header Host <bare-apex>;` — NEVER use this. The bare apex (e.g., `graj-media.com`) matches no server block on the home server. If you see this, it's almost certainly a copy-paste or manual-entry error.
|
||||
- **DuckDNS auto-detects source IP**: When the DuckDNS update script runs on the home server with `&ip=` (empty), DuckDNS uses the request's source IP — which is the HOME IP, not the VPS IP. For VPS reverse proxy setups, you MUST hardcode the VPS IP in the update URL: `&ip=51.81.84.34`. Otherwise the domain resolves to the home IP and bypasses the VPS entirely. Fix the cron script to always send the VPS IP.
|
||||
- **DuckDNS doesn't support dotted subdomains**: `immich.grajmedia.duckdns.org` won't resolve. Use hyphen format: `immich-grajmedia.duckdns.org`.
|
||||
- **DuckDNS auto-detects wrong IP**: The update script's `&ip=` parameter auto-detects the request's source IP. If the script runs on the home server but the domain should point to the VPS, DuckDNS sets the record to the home IP instead. **Fix:** always hardcode the VPS IP: `&ip=<VPS_IP>`. Script is at `/opt/duckdns/update.sh`. Verify with `dig +short <domain> @8.8.8.8`.
|
||||
- **/etc/hosts DNS override**: A local `/etc/hosts` entry like `127.0.0.1 grajmedia.duckdns.org` takes precedence over DNS. Useful for local access (avoids hairpin NAT), but causes confusion — `resolvectl query` shows "Data from: synthetic" when hosts file is active.
|
||||
- **certbot mangles configs on domain switch**: When migrating from DuckDNS to a new domain, certbot has already modified the nginx config files to add SSL directives AND secondary server blocks for HTTP→HTTPS redirect. Simply changing `server_name` values leaves broken configs (`listen` directives outside server blocks, missing certs). **The fix**: `sudo rm /etc/nginx/sites-enabled/*`, rewrite all configs from scratch, then re-run certbot for the new domain.
|
||||
- **Bash heredoc expansion in SSH commands**: Writing nginx configs via `ssh host "cat > file << 'EOF' ... $host ..."` may still expand `$host`, `$http_upgrade`, etc. because the double-quoted SSH wrapper triggers local expansion before the remote shell sees the heredoc. **Fix**: use single-quoted SSH: `ssh host 'cat > file << EOF ... $host ...'` (unquoted heredoc delimiter, but single-quoted SSH wrapper — the variable won't exist remotely so it stays literal), or patch the empty values afterwards with sed: `sudo sed -i 's|proxy_set_header Host ;|proxy_set_header Host \$host;|'`.
|
||||
- **Home nginx is SSL-only**: When proxying from VPS to home server nginx, you must use HTTPS with `proxy_ssl_verify off` and proper SNI headers. Direct Docker ports (HTTP) are simpler when available.
|
||||
- **127.0.0.1 Docker bindings**: Services bound to `127.0.0.1` are unreachable from Tailscale. Either bind to `0.0.0.0` or proxy through home nginx SSL. See `references/docker-port-binding.md`.
|
||||
- **certbot NXDOMAIN**: If certbot fails with NXDOMAIN, the DNS entry doesn't exist yet. Add the subdomain on DuckDNS first, wait for propagation, retry.
|
||||
- **OVH cloud images use `ubuntu` user by default**, not `root`. SSH keys go in `/home/ubuntu/.ssh/`, not `/root/.ssh/`. After key setup, use `sudo` to copy to root.
|
||||
- **Vaultwarden deployment**: Complete worked example — Docker Compose, home nginx with WebSocket support, VPS nginx, SSL setup, post-deploy steps. See `references/vaultwarden.md`.\n- **Plex Media Server**: Host-networked Plex proxied through VPS — large file streaming, WebSocket, 401 auth flow. See `references/plex.md`.\n
|
||||
- **Permanent certificate migration for Docker services**: When a Docker service like Vaultwarden uses Let's Encrypt certs managed by the host nginx (not by the container), the container doesn't need its own cert volume. Just bind-mount the data directory and let nginx handle SSL.
|
||||
- **DuckDNS auto-IP picks the wrong server**: When running the DuckDNS update script on the home server, leaving the `&ip=` parameter empty makes DuckDNS auto-detect the request's source IP — which is the HOME IP, not the VPS. The domain then resolves to the home server directly, bypassing the VPS entirely. **Fix**: hardcode the VPS IP in the update URL: `&ip=51.81.84.34`. The cron script at `/opt/duckdns/update.sh` must use the explicit VPS IP, not auto-detect.
|
||||
- **Let's Encrypt rate limits**: Certbot has a 5-certificates-per-domain-per-week limit. Bundle all subdomains into one cert (comma-separated `-d` flags) rather than separate certs per service.
|
||||
- **nginx fails at boot with 'host not found in upstream'**: nginx resolves all upstream hostnames at startup. If DNS isn't ready yet (common at boot, especially with `resolv.conf` pointing to `127.0.0.53` and systemd-resolved still initializing), nginx refuses to start — and ONE bad upstream takes down ALL sites. Fix: use runtime DNS resolution with `resolver` + variable. See `references/nginx-runtime-dns.md` for the pattern and worked example.
|
||||
- **Corporate firewall NRD (Newly Registered Domain) blocking**: Domains registered less than 30–60 days ago are blocked by most corporate web filters (Cisco Umbrella, FortiGuard, etc.) under the "Newly Registered and Observed Domains" category. This blocks ALL subdomains (`*.graj-media.com`) — the server config is fine, the block happens at the corporate DNS/firewall layer. No server-side fix exists. Workarounds: (1) Use an older well-established domain (like a years-old DuckDNS subdomain) as a temporary alias pointing to the same VPS — the NRD filter won't flag it. (2) Access directly via Tailscale IP (`https://100.93.253.36`) from work machines with Tailscale installed. (3) Ask corporate IT for a domain whitelist exception. The block lifts automatically once the domain ages past the NRD window (typically 30 days post-registration). See `references/duckdns-temp-alias.md` for the full temp-alias setup.
|
||||
- **Host header mismatch → SPA redirect loop**: When the VPS `proxy_pass` sends a `Host` header that doesn't match the home server nginx `server_name`, the request lands on the wrong server block (or the default). In SPAs with auth (ShopProQuote, etc.), this manifests as an infinite redirect between `login.html` and `index.html` — the auth check fails because API calls hit a different nginx context, the cookie/localStorage domain doesn't match, or the default server block serves unexpected content. See `references/spa-redirect-loop.md` for the full diagnostic workflow, worked example, and three fix patterns (file rename + reference update, server alias, stub page cache-breaker). **Two fixes, use the one that fits: (A) Add the VPS domain as a server alias on the home nginx:** `server_name grajmedia.duckdns.org <vps-domain1> <vps-domain2>;` then `sudo nginx -t && sudo systemctl reload nginx`. This preserves the original Host header through the chain. **(B) Match the Host header to what home nginx expects:** `proxy_set_header Host grajmedia.duckdns.org;` in the VPS nginx config. Option A is preferred when you have multiple services — it keeps Host-based routing intact. **Always verify** the home nginx config's `server_name` and ensure the Host header reaching it resolves to the right server block. **Browser cache trap:** after all server-side fixes are applied, the user may STILL see the stale redirect because their browser cached the 302 from the old setup. Verify with `curl -skI https://domain/` — if the server returns 200 but the browser shows a redirect, the fix is a hard refresh (Ctrl+Shift+R) or clearing site data in DevTools. A stub page at the old redirect target with a `<meta http-equiv="refresh">` to the correct page + no-cache headers breaks the cached redirect without user intervention.
|
||||
Reference in New Issue
Block a user