ray
78 lines
2.8 KiB
Markdown
78 lines
2.8 KiB
Markdown
# Hermes Local Model Configuration (Ollama)
|
|
|
|
When using local models via Ollama with Hermes Agent, the main model and each auxiliary task must be configured separately. This reference covers the full wiring.
|
|
|
|
## Quick Check
|
|
|
|
```bash
|
|
grep -A3 'provider:' ~/.hermes/config.yaml | grep -B1 'provider:'
|
|
```
|
|
|
|
This shows every provider setting. Any still pointing at a cloud provider that should be local needs updating.
|
|
|
|
## Main Model
|
|
|
|
```bash
|
|
hermes config set model.provider ollama
|
|
hermes config set model.default qwen2.5:14b
|
|
hermes config set model.base_url http://localhost:11434
|
|
```
|
|
|
|
## Auxiliary Sections
|
|
|
|
Each uses a model and must be configured independently. Changing `model.provider` does NOT affect them.
|
|
|
|
| Section | Purpose | Priority |
|
|
|---------|---------|----------|
|
|
| `auxiliary.vision` | Image analysis | High |
|
|
| `auxiliary.delegation` | Subagent chat model | High |
|
|
| `auxiliary.web_extract` | Web page summarization | Medium |
|
|
| `auxiliary.compression` | Context window compression | Medium |
|
|
| `auxiliary.approval` | Smart command approval | Low |
|
|
| `auxiliary.mcp` | MCP server execution | Low |
|
|
| `auxiliary.curator` | Skill lifecycle | Low |
|
|
| `auxiliary.title_generation` | Session naming | Low |
|
|
| `auxiliary.skills_hub` | Skills catalog search | Low |
|
|
| `auxiliary.kanban_decomposer` | Kanban task breakdown | Low |
|
|
| `auxiliary.profile_describer` | Profile descriptions | Low |
|
|
| `auxiliary.triage_specifier` | Model request routing | Low |
|
|
|
|
### Per-section config pattern
|
|
|
|
```bash
|
|
hermes config set auxiliary.vision.provider ollama
|
|
hermes config set auxiliary.vision.model llava:13b
|
|
hermes config set auxiliary.vision.base_url http://localhost:11434
|
|
|
|
hermes config set delegation.provider ollama
|
|
hermes config set delegation.model qwen2.5:14b
|
|
hermes config set delegation.base_url http://localhost:11434
|
|
```
|
|
|
|
Repeat for each auxiliary section as needed. Low-priority ones can safely stay on the default cloud provider for speed.
|
|
|
|
## Switching Back to Cloud
|
|
|
|
```bash
|
|
hermes config set model.provider deepseek
|
|
hermes config set model.default deepseek-v4-pro
|
|
```
|
|
|
|
## Gateway Restart
|
|
|
|
Config changes require a restart:
|
|
|
|
```bash
|
|
hermes gateway restart
|
|
```
|
|
|
|
**Pitfall:** `hermes gateway restart` is blocked inside the gateway process to prevent restart loops. The user must run it from their own shell terminal.
|
|
|
|
## Pitfalls
|
|
|
|
- Auxiliary models not switched — changing main model does not touch vision/compression/delegation.
|
|
- VRAM exhaustion — loading vision model while LLM is loaded can OOM. Ollama unloads idle models after 5 min.
|
|
- Speed expectations — local 14B is slower than cloud API. First cold-load takes 10-30 seconds.
|
|
- Config changes need `/reset` or relaunch — they don't hot-reload mid-session.
|
|
- Small cloud models may outpace local ones for latency-sensitive tasks (web extraction, compression).
|