Hermes — Overview

🜂 Hermes — Overview

Omair's always-on personal AI agent on the Oracle VPS. One-pager. Full detail → Hermes — Reference.

What it is

A self-hosted AI assistant you talk to over WhatsApp self-chat (text or voice, in and out), running 24/7 on oraclevps — with a web dashboard, scheduled automations, Google/ClickUp/web tools, an Obsidian knowledge vault, full self-monitoring, and a self-improvement loop. Remembers you via mem0.

What it can do

• Chat over WhatsApp — text or voice in; replies in text or voice (Gemini TTS, voice Kore).
• Capture — note:/todo:/idea:/... or a voice note → saved to your Obsidian Inbox.md (voice transcribed: Groq → Deepgram → Gemini → local Whisper).
• Morning brief (11 AM PKT) — weather, prayer, calendar, Google + ClickUp tasks, news → WhatsApp + email + Discord.
• Dedicated Discord feeds — #briefings, #alerts, #status, #news (tech + AI, 2×/day), #audit (every action), #seggy-said / #yasar-ahmad (creator updates), #trg-watch, #health.
• Self-monitoring — watchdogs probe every dependency every 5 min (services, LLMs, disk, dashboard uptime) and push to your phone (ntfy) + Discord. Healthchecks.io catches total outages.
• Self-improvement (Dojo) — daily 6AM cycle that analyzes session history, finds weaknesses, and patches its own skills automatically.
• Langfuse observability — every LLM call traced (tokens, cost, latency, tool invocations) → cloud.langfuse.com.
• Creator books → Obsidian — auto-follows Seggy Said & Yasar Ahmad, turning their clips into a categorized, tappable reading library (synced to phone + published at vault.omair.pages.dev).
• Session logging — every interactive session auto-logged to Obsidian with stats, tools used, key exchanges, and wikilinks.
• Google + ClickUp + web — Calendar, Gmail, Drive, Docs, Sheets, Slides, Tasks, Contacts; ClickUp; Tavily search, Context7 docs, Composio (1,000+ SaaS apps).
• Syncthing + Git backup — Obsidian vault synced to phone in real-time and committed to GitHub hourly for time-machine history.

Installed systems

Headlines

• Brain: GLM-5.2 (z.ai) + a 7-deep fallback chain (glm-5.1 → 5-turbo → 4.7 → 4.5-air → Gemini → Cerebras → Groq). All endpoints verified green 2026-06-25.
• Voice: STT via Groq (working) + Deepgram + Gemini. TTS via Gemini (voice Kore).
• Everything under ~/.hermes/; dashboard via ssh -L 9119 oraclevps → http://localhost:9119.
• Vault: ~/obsidian-vault/ synced via Syncthing + hourly git; web viewer at vault.omair.pages.dev.

Architecture Diagram (High-Level)

graph TB
    %% User platforms
    Omair["👤 Omair
Text + Voice"]
    WA["💬 WhatsApp
Bridge :3000"]
    DC["🎮 Discord
11 channels"]
    EM["📧 Email
IMAP/Gmail"]
    NT["🔔 ntfy
Push alerts"]

    %% Core
    GW["🜂 Hermes Gateway
Agent core · Session DB · Hooks"]
    LLM["🧠 LLM: GLM-5.2 (z.ai)
→ 5.1 → 5-turbo → 4.7
→ 4.5-air → Gemini → Cerebras → Groq"]

    %% MCP
    MCP["🔌 MCP Servers (6)
Google Suite · ClickUp · Composio
Context7 · Local Svc · DocIntel"]

    %% Memory
    MEM["💾 Memory
mem0 · MEMORY.md · USER.md
Session DB (SQLite)"]

    %% Plugins
    PLG["🧩 Plugins (6)
Audit · Capture · Langfuse
Security · Cleanup · Tavily"]

    %% Scheduled
    CRON["⏰ Scheduled Jobs
10 crontab + 9 Hermes cron
Brief · Watchdog · TRG · Dojo"]

    %% Voice
    VOICE["🎙️ Voice Pipeline
STT: Groq→Deepgram→Gemini→Whisper
TTS: Gemini (Kore)"]

    %% Outputs
    VAULT["📚 Obsidian Vault
Syncthing→Phone · Git→GitHub
vault.omair.pages.dev"]
    CF["🌐 Cloudflare Pages
Dashboard · Books · Transcripts
omair.pages.dev"]
    MON["🛡️ Monitoring
Watchdog · Langfuse · Audit
→ ntfy + Discord alerts"]

    %% External
    ZAI["z.ai / Gemini / Cerebras / Groq"]
    GAPI["Google APIs"]
    CUP["ClickUp · Composio"]

    %% Connections
    Omair --> WA & DC & EM & NT
    WA & DC & EM --> GW
    NT -.->|alerts| Omair
    GW <-->|reasoning| LLM
    LLM -->|API| ZAI
    GW -->|tools| MCP
    MCP --> GAPI & CUP
    GW -->|recall/store| MEM
    GW -.->|hooks| PLG
    GW <-->|STT/TTS| VOICE
    CRON -->|trigger| GW
    CRON -->|daily notes| VAULT
    GW -->|deploy| CF
    CRON -->|monitor| MON
    MON -.->|fail/recover| NT & DC

    classDef user fill:#083344,stroke:#22d3ee,color:#fff
    classDef core fill:#064e3b,stroke:#34d399,color:#fff
    classDef mcp fill:#4c1d95,stroke:#a78bfa,color:#fff
    classDef sched fill:#78350f,stroke:#fbbf24,color:#fff
    classDef plg fill:#881336,stroke:#fb7185,color:#fff
    classDef ext fill:#1e293b,stroke:#94a3b8,color:#fff
    classDef out fill:#083344,stroke:#22d3ee,color:#fff

    class Omair,WA,DC,EM,NT user
    class GW,LLM core
    class MCP,MEM mcp
    class CRON sched
    class PLG,MON plg
    class ZAI,GAPI,CUP ext
    class VAULT,CF out
    class VOICE out