# Ollama Deployment (Alternative to standalone llama-server)

Ollama provides on-demand model loading with automatic 5-minute idle unload.
Unlike standalone llama-server which keeps a model resident in VRAM permanently,
Ollama loads on first request and frees VRAM when idle — ideal for shared GPU
scenarios (work + inference on the same card).

## Installation

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

Creates systemd service `ollama`, listens on `127.0.0.1:11434`.

## Pull a model

```bash
# Text inference
ollama pull qwen2.5:7b

# Vision (image understanding)
ollama pull llava:7b
```

Models auto-unload after 5 minutes idle. `ollama ps` shows currently loaded models.

## Nginx reverse proxy — ⚠️ CRITICAL: Host header

**Ollama rejects requests where the `Host` header is not `localhost`.** When
proxying through nginx, you MUST override the Host header:

```nginx
location /llm/ {
    proxy_pass http://127.0.0.1:11434/;
    proxy_http_version 1.1;
    proxy_set_header Host "localhost";    # ← REQUIRED — Ollama rejects $host
    proxy_set_header X-Real-IP $remote_addr;
    proxy_buffering off;
    proxy_read_timeout 120s;              # allow for cold-start model load
}
```

Without this, nginx forwards the original Host header (`grajmedia.duckdns.org`,
`192.168.50.98`, etc.) and Ollama returns **403 Forbidden** immediately
(microsecond reject — not a model-loading delay).

Same fix applies for vision proxy:

```nginx
location /vision/ {
    proxy_pass http://127.0.0.1:11434/;
    proxy_http_version 1.1;
    proxy_set_header Host "localhost";
    proxy_set_header X-Real-IP $remote_addr;
    proxy_buffering off;
    proxy_read_timeout 300s;              # vision models load slower
}
```

## OpenAI-compatible endpoint

Ollama serves `/v1/chat/completions` with the same response format as
llama-server and the OpenAI API:

```json
{
  "id": "chatcmpl-898",
  "object": "chat.completion",
  "created": 1781059955,
  "model": "qwen2.5:7b",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "Hi!" },
    "finish_reason": "stop"
  }],
  "usage": { "prompt_tokens": 34, "completion_tokens": 3, "total_tokens": 37 }
}
```

## Migrating from llama-server to Ollama

llama-server does not require a `model` field in the request body. Ollama does.
When migrating, add the model name to every API call:

**Before (llama-server):**
```javascript
body: JSON.stringify({
    messages: [...],
    temperature: 0,
    max_tokens: 500
})
```

**After (Ollama):**
```javascript
body: JSON.stringify({
    model: 'qwen2.5:7b',
    messages: [...],
    temperature: 0,
    max_tokens: 500
})
```

Then update the nginx proxy target and restart:

```bash
# Change proxy_pass in nginx config, then:
sudo nginx -t && sudo systemctl reload nginx
```

Stop the old llama-server to free VRAM:

```bash
kill <llama-server-pid>
```

## Vision model API

Ollama's vision API uses the native chat format with an `images` array.
Base64 prefix `data:image/png;base64,` is NOT included — just the raw base64.

```javascript
const b64 = await new Promise(resolve => {
    const r = new FileReader();
    r.onload = () => resolve(r.result.split(',')[1]);  // strip data:image/...;base64,
    r.readAsDataURL(file);
});

const resp = await fetch('/vision/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
        model: 'llava:7b',
        messages: [{
            role: 'user',
            content: 'Your system prompt + instructions + schema',
            images: [b64]
        }],
        stream: false
    })
});
const data = await resp.json();
const content = data.message.content;  // note: not choices[0].message (that's /v1/chat/completions format)
```

**Key difference:** The native `/api/chat` endpoint returns `data.message.content`.
The OpenAI-compatible `/v1/chat/completions` returns `data.choices[0].message.content`.
Use `/v1/chat/completions` for text models and `/api/chat` for vision if you use
the native format — but send the `model` field either way.

## client_max_body_size

Vision requests include a base64 image which can be 2-5 MB (or more for large
screenshots). nginx default `client_max_body_size` is 1 MB — requests over this
are silently rejected with 413. Set in the server block:

```nginx
client_max_body_size 50m;
```

## Replacing Tesseract OCR with a vision model

For complex table layouts (schedules, invoices, spreadsheets), Tesseract cannot
reliably preserve row/column structure. A VLM (vision-language model) like
llava:7b sees the layout visually and outputs structured JSON directly,
eliminating the need for:
- Image preprocessing (upscaling, sharpening, contrast)
- TSV bounding-box parsing
- Coordinate-based row clustering
- OCR text → LLM text → JSON pipeline

**Old pipeline:** Screenshot → preprocessing → Tesseract → TSV bbox clustering →
text LLM → JSON

**New pipeline:** Screenshot → base64 → VLM → JSON