ray
71 lines
2.9 KiB
Markdown
71 lines
2.9 KiB
Markdown
# Hybrid VPS + Home LLM Hosting Pattern
|
|
|
|
When migrating a web app that has hardcoded local LLM calls to a VPS, keep the LLM at home and proxy calls back through Tailscale. The app code doesn't change — only the nginx routing changes.
|
|
|
|
## When to use
|
|
|
|
- The app makes `fetch('/llm/...')` calls hardcoded to a local Ollama endpoint
|
|
- The LLM requires a GPU (e.g., qwen2.5:14b at ~9GB VRAM) — a $5/mo VPS can't run it
|
|
- You want to move the frontend and database to a VPS (static IP, no home IP exposure)
|
|
- Home server stays on 24/7 anyway (Home Assistant, Immich, other services)
|
|
|
|
## Architecture
|
|
|
|
```
|
|
VPS ($5/mo) Home server (Tailscale)
|
|
├── Frontend (SPA) ├── Ollama (port 11434)
|
|
├── PocketBase / SQLite └── Tailscale IP: 100.xx.xx.xx
|
|
├── Nginx (SSL)
|
|
│ ├── / → frontend
|
|
│ ├── /api/ → PocketBase (local on VPS)
|
|
│ └── /llm/ → Tailscale → home:11434
|
|
└── Tailscale
|
|
```
|
|
|
|
## Nginx Config (VPS)
|
|
|
|
```nginx
|
|
# Add to existing server block:
|
|
location /llm/ {
|
|
proxy_pass http://<home-tailscale-ip>:11434/;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_read_timeout 120s;
|
|
proxy_buffering off;
|
|
}
|
|
```
|
|
|
|
## Key properties
|
|
|
|
- **Zero code changes.** The app's `fetch('/llm/v1/chat/completions')` calls work unchanged — nginx handles the routing.
|
|
- **Latency:** +2-5ms Tailscale overhead. For non-streaming LLM API calls (50-500 tokens), the added latency is completely irrelevant — the LLM generation time (2-5 seconds) dominates.
|
|
- **Bandwidth:** LLM API responses are small (JSON, a few KB). Tailscale overhead is negligible.
|
|
- **Reliability:** If the home server goes down, `/llm/` calls fail gracefully (the app should already handle API errors). Frontend and database keep working on the VPS.
|
|
|
|
## Migration checklist
|
|
|
|
1. **VPS:** Install nginx, Docker (for PocketBase if needed), Tailscale
|
|
2. **Rsync frontend** to VPS
|
|
3. **Rsync database** (PocketBase `pb_data/`, SQLite files) to VPS
|
|
4. **Tailscale:** Authorize VPS on the tailnet, note home server's Tailscale IP
|
|
5. **Nginx:** Add `/llm/` proxy block pointing to home Tailscale IP
|
|
6. **SSL:** Let's Encrypt cert for the VPS domain
|
|
7. **DNS:** Point domain to VPS IP
|
|
8. **Test:** Hit `/llm/v1/models` to verify Ollama is reachable through the proxy
|
|
|
|
## Example: ShopProQuote
|
|
|
|
Real-world application (shop management SPA):
|
|
|
|
| Component | Size | Migrate? |
|
|
|---|---|---|
|
|
| Frontend (HTML/JS/CSS) | 110 MB | → VPS |
|
|
| PocketBase (SQLite) | 17 MB (380 KB DB) | → VPS |
|
|
| Ollama (qwen2.5:14b) | ~9 GB VRAM | → stays home |
|
|
| Nginx config | — | → recreate on VPS |
|
|
|
|
LLM endpoints used: priority analysis, AI Write, repair order suggestions, appointment OCR parsing. All via `fetch('/llm/v1/chat/completions')` with `model: 'qwen2.5:14b'`.
|
|
|
|
Downtime: ~10 minutes during DNS switch. Test on a subdomain first.
|