Hermes — Reference

🜂 Hermes — Reference

Canonical detailed reference for the Hermes setup. Keep updated when the setup changes. Summary → Hermes — Overview.

High-level diagram: Hermes — Overview#Architecture Diagram (High-Level)

Architecture Diagram (Detailed)

graph TB
    subgraph VPS["Oracle Cloud VPS (oraclevps) · ARM Ampere · Ubuntu · SSH only"]
        direction TB

        subgraph Platforms["User Platforms"]
            WA["💬 WhatsApp
Bridge :3000 (Baileys)
self-chat"]
            DC["🎮 Discord Bot
#general #alerts #status
#news #audit #session-logs
#briefings #seggy-said
#yasar-ahmad #trg-watch #health"]
            EM["📧 Email
IMAP/SMTP Gmail
poll 15s"]
            NT["🔔 ntfy
push alerts"]
        end

        subgraph Core["Hermes Gateway"]
            GW["🜂 Gateway
hermes_cli.main gateway run
Agent core · Context compressor
Tool dispatch · Hooks"]
            SDB["Session DB
SQLite state.db
FTS5 search"]
            SK["Skills (60+)
15 categories
~/.hermes/skills/"]
            GW --- SDB
            GW --- SK
        end

        subgraph LLM["LLM Fallback Chain"]
            direction LR
            M1["glm-5.2
(primary)"]
            M2["glm-5.1"]
            M3["5-turbo"]
            M4["4.7"]
            M5["4.5-air"]
            M6["Gemini
2.5-flash"]
            M7["Cerebras
gpt-oss-120b"]
            M8["Groq
llama-3.3-70b"]
            M1 --> M2 --> M3 --> M4 --> M5 --> M6 --> M7 --> M8
        end

        subgraph Plugins["Plugins (6)"]
            PA["hermes-audit
post_tool_call
→ #audit + log"]
            PC["obsidian-capture
pre_gateway_dispatch
→ Inbox.md"]
            PL["langfuse
pre/post_llm_call
→ traces"]
            PS["security-guidance
various hooks"]
            PD["disk-cleanup
temp hygiene"]
            PT["web/tavily
search API"]
        end

        subgraph MCPS["MCP Servers (6)"]
            GS["google_suite
GCal·Tasks·Contacts
Drive·Sheets·Slides·Gmail"]
            CU["clickup
FastMCP
TEAM: 90181169347"]
            CO["composio
Hosted MCP
1000+ SaaS"]
            C7["context7
Live docs"]
            LS["local_services
Weather·Prayer
News·Maps"]
            DI["docintel
Doc extraction"]
        end

        subgraph MemLayer["Memory"]
            MM["mem0 cloud
MEM0_API_KEY
either/or w/ Honcho"]
            BI["MEMORY.md + USER.md
built-in · always active
5000/2500 chars"]
            SDB2["Session DB
SQLite · FTS5"]
        end

        subgraph Voice["Voice Pipeline"]
            STT["STT: Groq Whisper
→ Deepgram nova-2
→ Gemini
→ faster-whisper (CPU)"]
            TTS["TTS: Gemini Kore
+ Edge · OpenAI · ElevenLabs"]
        end

        subgraph Cron["Scheduled Jobs"]
            direction LR
            subgraph SC["System Crontab (10)"]
                MB["morning_brief.py
11AM PKT"]
                PR["prayer_reminder.py
every minute"]
                HW["hermes-watchdog.py
*/5 min"]
                HC["health_check.py
*/10 min"]
                NF["news_feed.py
9AM+9PM PKT"]
                OG["obsidian-git-backup.sh
hourly"]
                SL["session_logger.py
*/30 min"]
                DN["daily_note + backup + prune"]
            end
            subgraph HC2["Hermes Cron (9)"]
                TRG["TRG Watch
9AM+6PM"]
                DJ["Dojo Self-Improve
6AM daily"]
                SG["SeggySaid Monitor
6h"]
                YA["Yasar Monitor
6h"]
                DR["Dashboard Refresh
hourly"]
                FW["Fallback Watchdog
10m"]
                DU["Uptime Monitor
5m"]
                JID["JID Patch
6h"]
            end
        end

        subgraph Output["Outputs & Storage"]
            VAULT["📚 Obsidian Vault
~/obsidian-vault/
6 folders · wikilinks"]
            SYNC["Syncthing :8384
→ Samsung A55 (QUIC)"]
            GIT["Git → GitHub
hourly commit"]
            VW["Vault Watcher
polls 30s → wrangler"]
            CF["🌐 Cloudflare Pages
omair.pages.dev
Dashboard · Books · Transcripts"]
            DASH["Dashboard :9119
systemd · localhost"]
        end

        subgraph MonLayer["Monitoring & Observability"]
            WDG["hermes-watchdog
13 probes · */5min
state-change alerts"]
            LFE["Langfuse
LLM tracing · costs · latency"]
            AUD["hermes-audit
every tool call
→ #audit + audit.log"]
            FBW["Fallback Watchdog
model degradation
→ WhatsApp"]
            SLO["Session Logger
DB → Obsidian + Discord"]
            HCI["healthchecks.io
dead-man's-switch
5min period, 15min grace"]
        end
    end

    subgraph Ext["External Services (Internet)"]
        ZAI["z.ai API
glm-5.2 + fallbacks"]
        GAP["Google APIs
OAuth (gauth.py)"]
        CUP["ClickUp API
Composio Cloud"]
        GRO["Groq Whisper API"]
        DEEP["Deepgram nova-2"]
        GTTS["Gemini TTS API"]
        LFC["Langfuse Cloud
cloud.langfuse.com"]
        MEMC["mem0 Cloud"]
    end

    %% Platform → Gateway
    WA & DC & EM ==> GW
    GW -.->|responses| WA & DC & EM

    %% Gateway ↔ LLM
    GW <==>|prompt/completion| M1
    M1 & M6 & M7 & M8 -->|HTTPS| ZAI
    M6 & M8 --> GRO
    M6 --> GTTS

    %% Gateway → MCP
    GW ==>|tool calls| GS & CU & CO & C7 & LS & DI
    GS -->|OAuth| GAP
    CU -->|API| CUP
    CO -->|hosted| CUP

    %% Gateway → Memory
    GW <==>|recall/store| MM & BI & SDB2
    MM -->|HTTPS| MEMC

    %% Plugins → Gateway (hooks)
    PA & PC & PL & PS & PD & PT -.->|hooks| GW

    %% Voice
    GW <==>|audio in/out| STT & TTS
    STT --> GRO & DEEP
    TTS --> GTTS

    %% Cron → Gateway + Outputs
    MB & PR & HW & HC & NF -->|deliver| GW
    SL -->|logs| VAULT
    OG -->|commit| GIT
    TRG & DJ & SG & YA -->|trigger| GW
    DR -->|deploy| CF

    %% Monitoring
    HW -->|alerts| NT & DC
    FBW -->|alert| WA
    PL -.->|traces| LFE
    LFE -->|HTTPS| LFC
    HW -->|ping| HCI
    SL -->|embeds| DC

    %% Outputs
    GW -->|deploy| CF
    VAULT --- SYNC & GIT
    VAULT --> VW
    VW -->|redeploy| CF
    GW --> DASH

    %% Styling
    classDef user fill:#083344,stroke:#22d3ee,color:#fff
    classDef core fill:#064e3b,stroke:#34d399,color:#fff
    classDef mcp fill:#4c1d95,stroke:#a78bfa,color:#fff
    classDef sched fill:#78350f,stroke:#fbbf24,color:#fff
    classDef plg fill:#881336,stroke:#fb7185,color:#fff
    classDef ext fill:#1e293b,stroke:#94a3b8,color:#fff
    classDef out fill:#083344,stroke:#22d3ee,color:#fff
    classDef mon fill:#881336,stroke:#fb7185,color:#fff
    classDef voice fill:#431407,stroke:#fb923c,color:#fff

    class WA,DC,EM,NT user
    class GW,SDB,SK,M1,M2,M3,M4,M5,M6,M7,M8 core
    class GS,CU,CO,C7,LS,DI,MM,BI,SDB2 mcp
    class MB,PR,HW,HC,NF,OG,SL,DN,TRG,DJ,SG,YA,DR,FW,DU,JID sched
    class PA,PC,PL,PS,PD,PT plg
    class STT,TTS voice
    class VAULT,SYNC,GIT,VW,CF,DASH out
    class WDG,LFE,AUD,FBW,SLO,HCI mon
    class ZAI,GAP,CUP,GRO,DEEP,GTTS,LFC,MEMC ext

1. Host & access

• oraclevps (193.123.86.131, Oracle ARM/Ampere, Ubuntu). ssh oraclevps (user ubuntu).
• Everything under ~/.hermes/. Framework source: ~/.hermes/hermes-agent/.
• Externally only SSH(22) is reachable (host iptables default-accept except 3389 DROP; OCI security list blocks the rest).

2. Core services (systemd)

3. LLM (brain) — all healthy

• Primary: glm-5.2, provider custom, https://api.z.ai/api/coding/paas/v4 (Z_API_KEY).
• Fallback chain: glm-5.1 → glm-5-turbo → glm-4.7 → glm-4.5-air (z.ai) → gemini-2.5-flash → cerebras gpt-oss-120b → groq llama-3.3-70b-versatile.
• z.ai siblings sit on top because they tolerate GLM's reasoning_content field that Cerebras/Groq reject (HTTP 400).
• All 8 endpoints verified green (2026-06-25). Groq now reachable both directly and via WARP proxy (ASN ban appears lifted). Cerebras key renewed. Gemini responsive (no current quota issues).

4. Voice

• STT (in): stt.provider: groq — working off-CPU. Groq Whisper-turbo via WARP proxy (or direct). Voice-note capture order: Groq Whisper-turbo → Deepgram nova-2 → Gemini → local faster-whisper (keyless last-resort + creator-video transcriber). Deepgram + Groq both verified 200.
• TTS (out): tts.provider: gemini (gemini-3.1-flash-tts-preview, voice Kore). Replies sent as native WhatsApp voice bubbles via bridge /send-media (auto ogg/opus). Also configured: edge (AriaNeural), openai (gpt-4o-mini-tts, alloy), elevenlabs (multilingual_v2).
• Groq WARP history: Groq formerly returned Cloudflare error 1010 (Oracle ASN banned). Fixed by routing through WARP socks5 proxy (warp-svc, 127.0.0.1:40000) via hermes-groq-warp.service → ~/.hermes/hermes-groq-warp-proxy.py (listens :8788). Config groq.base_url + .env GROQ_BASE_URL → http://127.0.0.1:8788/openai/v1. As of 2026-06-25, Groq is also reachable directly (200) — WARP proxy kept as redundancy.

5. Messaging

• WhatsApp (primary): self-chat via bridge.js (:3000). Media → ~/.hermes/audio_cache. Send programmatically: POST http://localhost:3000/send {"chatId","message"}.
• Discord: bot with Manage Channels. Per-aspect routing: #general (chat), #briefings (morning brief), #alerts (watchdog fail), #status (watchdog recover + health_check), #news (news_feed 9am/9pm PKT), #audit (every tool call via hermes-audit plugin), #session-logs (session summaries via session_logger), #seggy-said / #yasar-ahmad (creator_notify), #trg-watch, #health.
• Email: IMAP/SMTP via gateway adapter (Gmail).
• ntfy: alert channel (watchdog, independent of WhatsApp).
• Config templates exist for Telegram, Slack, Matrix, Mattermost — not connected.

6. Memory

• memory.provider: mem0 (MEM0_API_KEY), enabled + user_profile, ~5000/2500 char budgets, plugin hermes-agent/plugins/memory/mem0/. Manage: hermes_cli.main memory .... External provider is either/or (cannot stack Honcho + mem0). Built-in MEMORY.md / USER.md always run alongside.

7. Scheduled jobs (system crontab, UTC)

8. Hermes cron jobs

9. MCP servers (config mcp_servers)

10. Custom in-house adapters (~/.hermes/custom-mcp/, venv .venv)

In-house Python over Google's official APIs: google_calendar.py, google_contacts.py, google_drive.py, google_sheets.py, google_slides.py, google_tasks.py, google_suite.py (unified), weather.py, prayer.py, news.py, maps.py, docintel.py, clickup.py, local_services.py, gauth.py (shared OAuth). Each has a matching .sh launcher wrapper (e.g. gcal.sh, gweather.sh, gnews.sh, gcontacts.sh, gsheets.sh, gtasks.sh, gslides.sh, gprayer.sh, gdrive.sh, gmaps.sh). Also: morning_brief.py, prayer_reminder.py, health_check.py, news_feed.py, obsidian_daily_note.py. Secrets in ~/.hermes/.env.

11. Custom scripts (~/.hermes/scripts/)

12. Plugins & hooks

12. Langfuse observability

• Cloud-hosted at cloud.langfuse.com (keys: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL).
• Traces every LLM call: input/output tokens, reasoning tokens, latency, tool invocations, model used, cost per call, cache hit/miss.
• Enabled via plugins.enabled: ["observability/langfuse"]. No separate systemd service — hooks into the agent runtime.
• Dashboard: https://cloud.langfuse.com → project traces.

13. Hermes Dojo (self-improvement)

• Skill: ~/.hermes/skills/hermes-dojo/ with scripts in scripts/ and data in data/.
• Cron job 394bbd064009 runs at 6AM PKT daily. Two-phase cycle: 1. Analyze — monitor.py scans session history, identifies top 3 weaknesses and skill gaps. 2. Fix — patches top weak skill via skill_manage(action="patch"), or creates a new skill if a capability was requested 3+ times.
• Output saved to ~/.hermes/cron/output/394bbd064009/. Delivers to WhatsApp (skips with [SILENT] if nothing to report).

14. Fallback Alert Watchdog

• Cron job 0fc364c13b32 (every 10min, no-agent script: scripts/fallback-watchdog.py).
• Monitors whether the primary model (glm-5.2) is active or if the system has fallen back to a cheaper model.
• Alerts to WhatsApp when fallback occurs — so Omair knows quality may be degraded.

15. Content pipeline (creator books)

• scripts/seggy_monitor.py / yas_ahmed_monitor.py: follow Seggy Said & Yasar Ahmad (YT/TikTok/FB) → yt-dlp download → faster-whisper transcribe → LLM essays → scripts/{seggy,yas_ahmed}-data/master.json (essay = markdown).
• build_html() → deploy_to_cf_pages → ~/hermes-portal/<book>/index.html + Cloudflare Pages (omair.pages.dev). Then generate_obsidian.py --book <book> → vault chapters.
• Batch scripts: yas_ahmed_batch_tt.py, yas_ahmed_batch_yt.py, yas_ahmed_summarize.py for bulk processing.
• Post-deploy: creator_notify.py → Discord #seggy-said / #yasar-ahmad.

16. Portal / dashboard (~/hermes-portal/, :9119)

dashboard.html (generate_dashboard.py), library/, seggy-book, yas-book, transcripts, generate_vault_viewer.py → omair.pages.dev/vault. Shared nav via shared_nav.py. Reach via ssh -L 9119 oraclevps.

17. Obsidian / Sync / Backup stack

• Syncthing ↔ phone "Omair Samsung A55" (direct QUIC); GUI 127.0.0.1:8384 (user omair).
• Git time-machine: vault git repo; ~/.local/bin/obsidian-git-backup.sh hourly → private GitHub omair-obsidian-vault. .stignore excludes .git + .obsidian/workspace*.
• Books: ~/.hermes/scripts/generate_obsidian.py → Books/<Author> — AI/ (Contents chip-list + chapters + Reading Dashboard).
• WhatsApp capture: plugin ~/.hermes/plugins/obsidian-capture/ (pre_gateway_dispatch) → ~/obsidian-vault/Inbox.md (text note:/todo:/idea:/inbox:/capture:/jot: or 📝; voice → transcribe → inbox).
• Daily notes: obsidian_daily_note.py (cron 5AM PKT) auto-generates Daily/YYYY-MM-DD.md from morning brief data.
• Session logs: session_logger.py (cron */30min) reads session DB → Session-Logs/ entries with stats, tools, key exchanges, vault links. Also posts compact embed to Discord #session-logs (source-colored).
• Dojo logs: ~/obsidian-vault/Dojo/ — one file per day (YYYY-MM-DD.md) with findings table, actions taken, skills modified, top performers. Index at Dojo/README.md. Cron writes log during overnight run + posts full report to Discord #dojo + sends brief WhatsApp summary.
• Web viewer: vault.omair.pages.dev — generate_vault_viewer.py builds static HTML, vault_watcher.service polls every 30s and redeploys via wrangler on change. Strips Dataview/Meta Bind/raw code blocks.

18. Skills library (~/.hermes/skills/)

~60 skills across 15 categories: - apple: Apple ecosystem integration - autonomous-ai-agents: claude-code, codex, custom-mcp-servers, hermes-agent, hermes-model-config, hermes-operations-dashboard, opencode - creative: architecture-diagram, ascii-art, ascii-video, baoyu-infographic, claude-design, comfyui, design-md, excalidraw, humanizer, manim-video, p5js, popular-web-designs, pretext, songwriting-and-ai-music, touchdesigner-mcp - data-science: jupyter-live-kernel - devops: cloudflare-pages-deploy, deployment, kanban-orchestrator, kanban-worker, systemd-ops - dogfood: exploratory QA of web apps - email: himalaya (IMAP/SMTP from terminal) - github: codebase-inspection, github-auth, github-code-review, github-issues, github-pr-workflow, github-repo-management - hermes-dojo: continuous self-improvement (see §13) - media: gif-search, omair-transcription-treatment, songsee, video-transcription, youtube-content - mlops: huggingface-hub, evaluation (lm-eval-harness, weights-and-biases), inference (llama-cpp, vllm), models (audiocraft, segment-anything) - note-taking: obsidian - productivity: airtable, google-calendar, google-docs-formatting, google-drive, hermes-cron-jobs, maps, market-data, memory-management, notion, ocr-and-documents, powerpoint, teams-meeting-pipeline, whatsapp-messaging - research: arxiv, blogwatcher, llm-wiki, polymarket, web-retrieval - smart-home: openhue (Philips Hue) - social-media: xurl (X/Twitter) - software-development: database-operations, git-operations, hermes-agent-skill-authoring, node-inspect-debugger, plan, python-debugpy, requesting-code-review, security-hardening, spike, systematic-debugging, test-driven-development - yuanbao: Yuanbao (元宝) Chinese platform groups

19. Keys (~/.hermes/.env)

Platform config vars (not secrets — operational config): | Key | Purpose | |---|---| | WHATSAPP_ENABLED / WHATSAPP_MODE / WHATSAPP_HOME_CHANNEL / WHATSAPP_HOME_CHANNEL_THREAD_ID / WHATSAPP_ALLOWED_USERS | WhatsApp bridge config | | DISCORD_HOME_CHANNEL / DISCORD_HOME_CHANNEL_THREAD_ID / DISCORD_ALLOWED_USERS / DISCORD_ALLOW_ALL_USERS | Discord bot config | | DISCORD_*_CHANNEL (ALERT, STATUS, AUDIT, NEWS, BRIEF, HEALTH, SEGGY, YAS) | Discord channel routing IDs | | EMAIL_ADDRESS / EMAIL_PASSWORD / EMAIL_IMAP_HOST / EMAIL_IMAP_PORT / EMAIL_SMTP_HOST / EMAIL_SMTP_PORT / EMAIL_POLL_INTERVAL / EMAIL_HOME_ADDRESS / EMAIL_ALLOWED_USERS | Gmail IMAP/SMTP config | | OBSIDIAN_VAULT_PATH | ~/obsidian-vault | | AGENT_BROWSER_EXECUTABLE_PATH | Browser binary for automation | | BROWSER_* (SESSION_TIMEOUT, INACTIVITY_TIMEOUT) / BROWSERBASE_* (PROXIES, ADVANCED_STEALTH) | Browser automation tuning | | TERMINAL_TIMEOUT / TERMINAL_LIFETIME_SECONDS / TERMINAL_MODAL_IMAGE | Terminal tool limits | | *_DEBUG (WEB, VISION, MOA, IMAGE) | Debug flags for tool subsystems |

20. Known issues / watch-list

• None critical. All LLM endpoints green, all STT providers green, zero failed systemd units (2026-06-25).
• WARP proxy (hermes-groq-warp.service) kept as redundancy — Groq now works directly too.
• OPENAI_API_KEY is intentionally the z.ai key, not a bug.
• Watch for: Gemini quota limits (intermittent 429s under heavy use), GLM-5.2 reasoning_content token overhead (max_tokens ≥ 8000 for essay-length output).

21. Monitoring & alerting stack

22. Content routing policy (three-pillar model)

WhatsApp = Personal & Time-Sensitive (~10 msgs/day max) - Morning Brief (full), Prayer Reminders, TRG Watch (1-line summary), Seggy/Yasar (1-line summary), Dojo summary, custom reminders, health PROBLEM alerts

Discord = Archive & Detail (all 12 channels) - #briefings: Morning Brief full copy - #trg-watch: TRG Watch full analysis - #seggy-said / #yasar-ahmad: Creator content full report - #alerts: Fallback alerts + watchdog failures - #status: Health heartbeats + recoveries - #news: News feed - #audit: Real-time tool call log - #session-logs: Session summaries - #dojo: Dojo self-improvement reports - #general: Interactive chat (future)

Obsidian Vault = Knowledge & History - Session-Logs/: Full session transcripts - Dojo/: Daily improvement logs - Daily/: Daily notes - Hermes/: System documentation - Inbox.md: Quick capture via obsidian-capture plugin

23. Reading library

• Books/<Author> — AI/<Category>/ chapters: cleaned titles, 2-pass dedup, Released/Platform chips, Prev/Next nav, ▶ source link, per-chapter Meta Bind status toggle.
• 📖 Reading Dashboard = static shelf (renders everywhere) + Dataview progress (Obsidian only).
• Web viewer strips ```dataview, Meta Bind , and hidden dirs (.stversions/.obsidian).
• Obsidian plugins required: Dataview + Meta Bind (installed).