---
name: hermes-local-providers
description: "Configure Hermes Agent to use self-hosted, local LLM providers (Ollama, vLLM, llama.cpp, etc.) via OpenAI-compatible custom endpoints."
version: 1.0.0
author: agent
created_by: agent
metadata:
  hermes:
    tags: [hermes, ollama, local-models, configuration, self-hosted, vllm, llm-serving]
    related_skills: [hermes-agent, llama-cpp, serving-llms-vllm, docker-gpu-acceleration]
---

# Hermes Local Providers

Configure Hermes Agent to use self-hosted / local LLMs that expose an OpenAI-compatible API endpoint.

## Supported Backends

| Backend | Typical Base URL | Auth | Notes |
|---------|-----------------|------|-------|
| Ollama | `http://localhost:11434/v1` | Placeholder | Most common for local use |
| vLLM | `http://localhost:8000/v1` | Optional API key | Production-scale serving |
| llama.cpp | `http://localhost:8080/v1` | None | Lightweight GGUF inference |

## Configuration

### 1. Set the custom provider

```bash
hermes config set model.provider custom
hermes config set model.base_url http://localhost:11434/v1
hermes config set model.api_key ollama
hermes config set model.default qwen2.5:14b
```

The provider name is **`custom`** — no suffix, no prefix. Hermes recognizes `custom` as a reserved provider that reads `model.base_url` and `model.api_key` directly from the config. Values like `custom:ollama` or `custom:vllm` will be rejected as "Unknown provider."

Ollama doesn't need a real API key, but `model.api_key` must be set to something non-empty. `ollama` (or any dummy string) works.

### 2. ⚠️ Verify the config persisted

`hermes config set` on `model.*` keys may report success (`✓ Set model.key = value`) but NOT actually write the value to `~/.hermes/config.yaml`. This is a known pitfall — always verify:

```bash
grep -A6 '^model:' ~/.hermes/config.yaml
```

Expected result:
```yaml
model:
  default: qwen2.5:14b
  provider: custom
  base_url: http://localhost:11434/v1
  api_key: ollama
```

> ⚠️ The provider must be **`custom`** (bare) — NOT `custom:ollama`, `custom:vllm`, or any suffix. Hermes only recognizes the bare `custom` as a reserved provider for user-defined endpoints. `custom:ollama` will be rejected as "Unknown provider."

If the old values are still showing, re-run the `hermes config set` commands and re-check. Retrying usually works; the silent failure appears to be intermittent.

### 3. Test the API independently

Before restarting the gateway, confirm the local backend is reachable:

```bash
curl -s http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen2.5:14b","messages":[{"role":"user","content":"Say hello in 3 words"}],"stream":false}'
```

If this fails, Ollama/vLLM/llama.cpp isn't running or the model isn't loaded.

### 4. Restart the gateway

```bash
hermes gateway restart
```

Config changes only take effect after a gateway restart. For CLI sessions, exit and relaunch.

## Pitfalls

- **Config set silent failure**: `hermes config set` for `model.*` keys (`provider`, `base_url`, `api_key`, `default`) may report "✓ Set" but not persist. Always grep the config file to confirm before assuming the change took effect.
- **Auth placeholder required**: A local Ollama endpoint doesn't authenticate, but Hermes requires `model.api_key` to be non-empty. Any dummy value works.
- **Gateway restart required**: Unlike some config changes, switching the model provider requires a full gateway restart. `/reset` or `/model` in-session won't pick up the new provider settings.
- **Auxiliary models stay unchanged**: Vision, compression, web extraction, title generation, and all other `auxiliary.*` subsystems keep their own provider config. Switching the main model to Ollama doesn't affect these — they'll continue using whatever provider/API key they were configured with (DeepSeek, OpenRouter, etc.).
- **Delegation has its own config**: The `delegation.*` section (`delegation.model`, `delegation.provider`, `delegation.base_url`) controls subagent models. To route subagents through the local provider too, set those separately:
  ```bash
  hermes config set delegation.provider custom
  hermes config set delegation.base_url http://localhost:11434/v1
  hermes config set delegation.api_key ollama
  hermes config set delegation.model qwen2.5:14b
  ```
- **Model must be downloaded**: `ollama list` shows installed models. If you set `model.default` to a model that isn't pulled, Ollama will pull it on first request (slow first call) or error if auto-pull is disabled.

## Verification

After restart, send a message to the agent. If you get "provider authentication error":

1. Check `~/.hermes/config.yaml` — the old provider values may have persisted
2. Re-run the `hermes config set` commands
3. Verify with `grep -A5 '^model:' ~/.hermes/config.yaml`
4. Restart gateway again

The "authentication error" is misleading — it usually means the API key/base URL weren't received by Hermes, which happens when `hermes config set` failed to persist.