initial commit

2026-07-12 10:17:17 -04:00
commit dab5a4ebc6
1424 changed files with 330463 additions and 0 deletions
@@ -0,0 +1,105 @@
+---
+name: hermes-local-providers
+description: "Configure Hermes Agent to use self-hosted, local LLM providers (Ollama, vLLM, llama.cpp, etc.) via OpenAI-compatible custom endpoints."
+version: 1.0.0
+author: agent
+created_by: agent
+metadata:
+  hermes:
+    tags: [hermes, ollama, local-models, configuration, self-hosted, vllm, llm-serving]
+    related_skills: [hermes-agent, llama-cpp, serving-llms-vllm, docker-gpu-acceleration]
+---
+
+# Hermes Local Providers
+
+Configure Hermes Agent to use self-hosted / local LLMs that expose an OpenAI-compatible API endpoint.
+
+## Supported Backends
+
+| Backend | Typical Base URL | Auth | Notes |
+|---------|-----------------|------|-------|
+| Ollama | `http://localhost:11434/v1` | Placeholder | Most common for local use |
+| vLLM | `http://localhost:8000/v1` | Optional API key | Production-scale serving |
+| llama.cpp | `http://localhost:8080/v1` | None | Lightweight GGUF inference |
+
+## Configuration
+
+### 1. Set the custom provider
+
+```bash
+hermes config set model.provider custom
+hermes config set model.base_url http://localhost:11434/v1
+hermes config set model.api_key ollama
+hermes config set model.default qwen2.5:14b
+```
+
+The provider name is **`custom`** — no suffix, no prefix. Hermes recognizes `custom` as a reserved provider that reads `model.base_url` and `model.api_key` directly from the config. Values like `custom:ollama` or `custom:vllm` will be rejected as "Unknown provider."
+
+Ollama doesn't need a real API key, but `model.api_key` must be set to something non-empty. `ollama` (or any dummy string) works.
+
+### 2. ⚠️ Verify the config persisted
+
+`hermes config set` on `model.*` keys may report success (`✓ Set model.key = value`) but NOT actually write the value to `~/.hermes/config.yaml`. This is a known pitfall — always verify:
+
+```bash
+grep -A6 '^model:' ~/.hermes/config.yaml
+```
+
+Expected result:
+```yaml
+model:
+  default: qwen2.5:14b
+  provider: custom
+  base_url: http://localhost:11434/v1
+  api_key: ollama
+```
+
+> ⚠️ The provider must be **`custom`** (bare) — NOT `custom:ollama`, `custom:vllm`, or any suffix. Hermes only recognizes the bare `custom` as a reserved provider for user-defined endpoints. `custom:ollama` will be rejected as "Unknown provider."
+
+If the old values are still showing, re-run the `hermes config set` commands and re-check. Retrying usually works; the silent failure appears to be intermittent.
+
+### 3. Test the API independently
+
+Before restarting the gateway, confirm the local backend is reachable:
+
+```bash
+curl -s http://localhost:11434/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model":"qwen2.5:14b","messages":[{"role":"user","content":"Say hello in 3 words"}],"stream":false}'
+```
+
+If this fails, Ollama/vLLM/llama.cpp isn't running or the model isn't loaded.
+
+### 4. Restart the gateway
+
+```bash
+hermes gateway restart
+```
+
+Config changes only take effect after a gateway restart. For CLI sessions, exit and relaunch.
+
+## Pitfalls
+
+- **Config set silent failure**: `hermes config set` for `model.*` keys (`provider`, `base_url`, `api_key`, `default`) may report "✓ Set" but not persist. Always grep the config file to confirm before assuming the change took effect.
+- **Auth placeholder required**: A local Ollama endpoint doesn't authenticate, but Hermes requires `model.api_key` to be non-empty. Any dummy value works.
+- **Gateway restart required**: Unlike some config changes, switching the model provider requires a full gateway restart. `/reset` or `/model` in-session won't pick up the new provider settings.
+- **Auxiliary models stay unchanged**: Vision, compression, web extraction, title generation, and all other `auxiliary.*` subsystems keep their own provider config. Switching the main model to Ollama doesn't affect these — they'll continue using whatever provider/API key they were configured with (DeepSeek, OpenRouter, etc.).
+- **Delegation has its own config**: The `delegation.*` section (`delegation.model`, `delegation.provider`, `delegation.base_url`) controls subagent models. To route subagents through the local provider too, set those separately:
+  ```bash
+  hermes config set delegation.provider custom
+  hermes config set delegation.base_url http://localhost:11434/v1
+  hermes config set delegation.api_key ollama
+  hermes config set delegation.model qwen2.5:14b
+  ```
+- **Model must be downloaded**: `ollama list` shows installed models. If you set `model.default` to a model that isn't pulled, Ollama will pull it on first request (slow first call) or error if auto-pull is disabled.
+
+## Verification
+
+After restart, send a message to the agent. If you get "provider authentication error":
+
+1. Check `~/.hermes/config.yaml` — the old provider values may have persisted
+2. Re-run the `hermes config set` commands
+3. Verify with `grep -A5 '^model:' ~/.hermes/config.yaml`
+4. Restart gateway again
+
+The "authentication error" is misleading — it usually means the API key/base URL weren't received by Hermes, which happens when `hermes config set` failed to persist.
@@ -0,0 +1,44 @@
+# Ollama "Provider Authentication Error" Fix
+
+## Symptom
+
+After configuring a local provider (Ollama, vLLM, etc.), the agent reports "Provider authentication failed" on the next request. Gateway logs show: `Unknown provider 'custom:<name>'`.
+
+## Root Cause
+
+**Wrong provider name.** The Hermes custom endpoint provider is registered as **`custom`** (bare), not `custom:ollama`, `custom:vllm`, or any `custom:<name>` variant. The suffix format is not recognized — `custom:ollama` and `openai` are both rejected as unknown providers.
+
+Secondary cause: `hermes config set` for `model.*` keys can silently fail to persist (reports "✓ Set" but file shows old values). Always verify with `grep -A5 '^model:' ~/.hermes/config.yaml`.
+
+## Fix
+
+1. Set the provider to `custom` (no suffix):
+   ```bash
+   hermes config set model.provider custom
+   hermes config set model.base_url http://localhost:11434/v1
+   hermes config set model.api_key ollama
+   hermes config set model.default qwen2.5:14b
+   ```
+
+2. **Verify the file**:
+   ```bash
+   grep -A6 '^model:' ~/.hermes/config.yaml
+   ```
+   Must show `provider: custom`.
+
+3. Test the Ollama API directly:
+   ```bash
+   curl -s http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" \
+     -d '{"model":"qwen2.5:14b","messages":[{"role":"user","content":"hi"}],"stream":false}'
+   ```
+
+4. Restart gateway (from OUTSIDE the gateway process — cannot restart from within):
+   ```bash
+   hermes gateway restart
+   ```
+
+## Notes
+
+- The provider `custom` is a reserved Hermes provider that reads `model.base_url` and `model.api_key` directly. It uses `transport="openai_chat"` (OpenAI-compatible API).
+- If you set `custom` and still get the error, check with `grep` — the `config set` may have silently failed. Re-run the commands.
+- Gateway restart must be done from a separate shell (SSH, desktop terminal) — not from within a gateway session.