ray
2.8 KiB
2.8 KiB
Hermes Local Model Configuration (Ollama)
When using local models via Ollama with Hermes Agent, the main model and each auxiliary task must be configured separately. This reference covers the full wiring.
Quick Check
grep -A3 'provider:' ~/.hermes/config.yaml | grep -B1 'provider:'
This shows every provider setting. Any still pointing at a cloud provider that should be local needs updating.
Main Model
hermes config set model.provider ollama
hermes config set model.default qwen2.5:14b
hermes config set model.base_url http://localhost:11434
Auxiliary Sections
Each uses a model and must be configured independently. Changing model.provider does NOT affect them.
| Section | Purpose | Priority |
|---|---|---|
auxiliary.vision |
Image analysis | High |
auxiliary.delegation |
Subagent chat model | High |
auxiliary.web_extract |
Web page summarization | Medium |
auxiliary.compression |
Context window compression | Medium |
auxiliary.approval |
Smart command approval | Low |
auxiliary.mcp |
MCP server execution | Low |
auxiliary.curator |
Skill lifecycle | Low |
auxiliary.title_generation |
Session naming | Low |
auxiliary.skills_hub |
Skills catalog search | Low |
auxiliary.kanban_decomposer |
Kanban task breakdown | Low |
auxiliary.profile_describer |
Profile descriptions | Low |
auxiliary.triage_specifier |
Model request routing | Low |
Per-section config pattern
hermes config set auxiliary.vision.provider ollama
hermes config set auxiliary.vision.model llava:13b
hermes config set auxiliary.vision.base_url http://localhost:11434
hermes config set delegation.provider ollama
hermes config set delegation.model qwen2.5:14b
hermes config set delegation.base_url http://localhost:11434
Repeat for each auxiliary section as needed. Low-priority ones can safely stay on the default cloud provider for speed.
Switching Back to Cloud
hermes config set model.provider deepseek
hermes config set model.default deepseek-v4-pro
Gateway Restart
Config changes require a restart:
hermes gateway restart
Pitfall: hermes gateway restart is blocked inside the gateway process to prevent restart loops. The user must run it from their own shell terminal.
Pitfalls
- Auxiliary models not switched — changing main model does not touch vision/compression/delegation.
- VRAM exhaustion — loading vision model while LLM is loaded can OOM. Ollama unloads idle models after 5 min.
- Speed expectations — local 14B is slower than cloud API. First cold-load takes 10-30 seconds.
- Config changes need
/resetor relaunch — they don't hot-reload mid-session. - Small cloud models may outpace local ones for latency-sensitive tasks (web extraction, compression).