ray
32 lines
1.0 KiB
Markdown
32 lines
1.0 KiB
Markdown
# Current Ollama Model Inventory
|
|
|
|
Host: RTX 2080 Ti FE (11GB VRAM)
|
|
Ollama on `127.0.0.1:11434`, nginx proxies `/llm/` and `/vision/` to it.
|
|
|
|
## Active Models
|
|
|
|
| Model | Size | Role | Used By |
|
|
|-------|------|------|---------|
|
|
| `qwen2.5:14b` | 9.0 GB (Q4_K_M) | Text LLM | AI Write, Priority Analysis, Generate Priorities |
|
|
| `llava:13b` | 8.0 GB (Q4_0) | Vision | Screenshot Scan (appointments.html) |
|
|
|
|
## Removed Models
|
|
|
|
- `qwen2.5:7b` (replaced by 14b)
|
|
- `moondream:latest` (redundant — llava:13b covers vision)
|
|
- `llava:7b` (replaced by 13b)
|
|
|
|
## VRAM Budget
|
|
|
|
- qwen2.5:14b loaded: ~9.7 GB (86% VRAM, ~1.6 GB headroom for context)
|
|
- llava:13b loaded: ~8.0 GB (~3 GB headroom)
|
|
- Both models can't load simultaneously on 11GB
|
|
- Auto-unload after 5 min idle (`OLLAMA_KEEP_ALIVE=5m`)
|
|
|
|
## Updating Models
|
|
|
|
1. `ollama pull <model>` to download
|
|
2. `ollama rm <old-model>` to clean up
|
|
3. Update hardcoded model name in `api.js` (lines 36 and 143) for LLM swaps
|
|
4. Vision dropdown in `appointments.html` auto-populates from Ollama — no HTML changes needed
|