hermes-config/skills/autonomous-ai-agents/hermes-local-providers/SKILL.md

name, description, version, author, created_by, metadata

name	description	version	author	created_by	metadata
hermes-local-providers	Configure Hermes Agent to use self-hosted, local LLM providers (Ollama, vLLM, llama.cpp, etc.) via OpenAI-compatible custom endpoints.	1.0.0	agent	agent	hermes tags related_skills hermes ollama local-models configuration self-hosted vllm llm-serving hermes-agent llama-cpp serving-llms-vllm docker-gpu-acceleration

Hermes Local Providers

Configure Hermes Agent to use self-hosted / local LLMs that expose an OpenAI-compatible API endpoint.

Supported Backends

Backend	Typical Base URL	Auth	Notes
Ollama	http://localhost:11434/v1	Placeholder	Most common for local use
vLLM	http://localhost:8000/v1	Optional API key	Production-scale serving
llama.cpp	http://localhost:8080/v1	None	Lightweight GGUF inference

Configuration

1. Set the custom provider

hermes config set model.provider custom
hermes config set model.base_url http://localhost:11434/v1
hermes config set model.api_key ollama
hermes config set model.default qwen2.5:14b

The provider name is custom — no suffix, no prefix. Hermes recognizes custom as a reserved provider that reads model.base_url and model.api_key directly from the config. Values like custom:ollama or custom:vllm will be rejected as "Unknown provider."

Ollama doesn't need a real API key, but model.api_key must be set to something non-empty. ollama (or any dummy string) works.

2. ⚠️ Verify the config persisted

hermes config set on model.* keys may report success (✓ Set model.key = value) but NOT actually write the value to ~/.hermes/config.yaml. This is a known pitfall — always verify:

grep -A6 '^model:' ~/.hermes/config.yaml

Expected result:

model:
  default: qwen2.5:14b
  provider: custom
  base_url: http://localhost:11434/v1
  api_key: ollama

⚠️ The provider must be custom (bare) — NOT custom:ollama, custom:vllm, or any suffix. Hermes only recognizes the bare custom as a reserved provider for user-defined endpoints. custom:ollama will be rejected as "Unknown provider."

If the old values are still showing, re-run the hermes config set commands and re-check. Retrying usually works; the silent failure appears to be intermittent.

3. Test the API independently

Before restarting the gateway, confirm the local backend is reachable:

curl -s http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen2.5:14b","messages":[{"role":"user","content":"Say hello in 3 words"}],"stream":false}'

If this fails, Ollama/vLLM/llama.cpp isn't running or the model isn't loaded.

4. Restart the gateway

hermes gateway restart

Config changes only take effect after a gateway restart. For CLI sessions, exit and relaunch.

Pitfalls

• Config set silent failure: hermes config set for model.* keys (provider, base_url, api_key, default) may report "✓ Set" but not persist. Always grep the config file to confirm before assuming the change took effect.
• Auth placeholder required: A local Ollama endpoint doesn't authenticate, but Hermes requires model.api_key to be non-empty. Any dummy value works.
• Gateway restart required: Unlike some config changes, switching the model provider requires a full gateway restart. /reset or /model in-session won't pick up the new provider settings.
• Auxiliary models stay unchanged: Vision, compression, web extraction, title generation, and all other auxiliary.* subsystems keep their own provider config. Switching the main model to Ollama doesn't affect these — they'll continue using whatever provider/API key they were configured with (DeepSeek, OpenRouter, etc.).
• Delegation has its own config: The delegation.* section (delegation.model, delegation.provider, delegation.base_url) controls subagent models. To route subagents through the local provider too, set those separately:

  hermes config set delegation.provider custom
  hermes config set delegation.base_url http://localhost:11434/v1
  hermes config set delegation.api_key ollama
  hermes config set delegation.model qwen2.5:14b
  

• Model must be downloaded: ollama list shows installed models. If you set model.default to a model that isn't pulled, Ollama will pull it on first request (slow first call) or error if auto-pull is disabled.

Verification

After restart, send a message to the agent. If you get "provider authentication error":

1. Check ~/.hermes/config.yaml — the old provider values may have persisted
2. Re-run the hermes config set commands
3. Verify with grep -A5 '^model:' ~/.hermes/config.yaml
4. Restart gateway again

The "authentication error" is misleading — it usually means the API key/base URL weren't received by Hermes, which happens when hermes config set failed to persist.