The CLI module provides real-time system observability for an eevee-bot deployment. It connects to the NATS messaging backbone, tracks module health and connector status, detects anomalies, and displays periodic summary tables — giving operators a live dashboard of the entire bot infrastructure.

A companion Docker image (the toolbox) packages the monitor into a lightweight Alpine container with the NATS CLI utility pre-installed, making it easy to run debugging sessions alongside your eevee workloads in Kubernetes.

Features

  • Real-time system observability — tracks module health, connector status, and registrations across the deployment
  • Periodic summary tables — 9-column ASCII table with version, uptime, message/command counts, errors, memory, and p50/p95 latency percentiles
  • Event detection — 12 event types covering module lifecycle, connector changes, registration activity, and anomalies
  • Anomaly detection — automatic restart detection, error delta tracking, stale module detection, connector channel count changes
  • Two renderers — follow mode (colored events + summary tables) and raw mode (unformatted NATS firehose)
  • CRD-driven configurationsummaryInterval, maxModuleAge, displayEvents, and filters configurable via YAML
  • Container-friendly — forces ANSI colors for kubectl logs / k9s
  • Stats reporting — monitor reports its own module/connector counts and event throughput

Usage

Follow Mode (default)

Displays real-time events as timestamped lines, plus a periodic summary table:

  eevee monitor
  

Example output:

  04:12:03 ▸ module appeared: weather
04:12:03 ▸ module appeared: router
04:12:08 ▸ connector-irc:libera reconnected
04:13:03 ▸ admin errors increased by 2 (0 → 2)

──────────────────────────────────────────────────────────
  eevee monitor · 04:13 UTC · 12 modules · 1 connector
──────────────────────────────────────────────────────────
  Module          Ver      Uptime   Msgs    Cmds  Errs    Mem   MsgLat     CmdLat
  ● admin         2.7.3     2h15m   1.2k      43     2   58MB  3/12ms    15/89ms
  ● router        2.5.5     2h15m   4.1k     210     0   72MB  1/5ms     8/34ms
  ○ seen          1.5.5     2h15m     86       5     0   42MB  —         —
  ● connector-irc 1.6.6     2h15m   3.8k       0     0   65MB  2/8ms     —
  ...

  IRC: libera ● connected  #eevee,#testing
──────────────────────────────────────────────────────────
  

Summary Table Columns

ColumnDescription
ModuleStatus dot + name (truncated at 15 chars)
VerModule version
UptimeTime since module started
MsgsTotal messages processed
CmdsTotal commands executed
ErrsTotal errors
MemRSS memory in MB
MsgLatMessage processing latency (p50/p95 in ms)
CmdLatCommand processing latency (p50/p95 in ms)

Status Dots

SymbolStatusMeaning
● (green)healthyRunning, no errors
● (yellow)degradedRunning, but has errors
○ (red)downNo recent stats (stale) or uptime is zero

Raw Mode

Dumps every NATS message unformatted — useful for low-level debugging:

  eevee monitor --raw
  

Example output:

  [stats.emit.response.weather] {"module":"weather","stats":{"version":"1.4.5",...}}
[command.register] {"module":"admin","command":"health"}
[control.connectors.irc.libera] {"action":"connect","network":"libera"}
  

Options

FlagDefaultDescription
--followonAppend-only stdout mode with colored events and periodic summary tables
--rawoffUnformatted NATS firehose — prints every message as [subject] payload
--filter <prefix>(none)Only observe messages matching this subject prefix
--modules <list>(all)Only track specified modules (comma-separated)
--no-summaryoffDisable periodic summary blocks (events still shown)
--no-coloroffStrip ANSI colors from output

Events

The monitor observes 12 event types across the NATS backbone:

EventDescription
module_startNew module appeared, or module recovered from down/degraded
module_stopModule went down or stopped responding
module_errorModule’s error count increased
connector_connectConnector appeared or status changed to connected
connector_disconnectConnector disconnected
connector_reconnectConnector reconnected after being disconnected
registrationCommand, broadcast, or help entry registered
unregistrationCommand, broadcast, or help entry removed
backup_startBackup operation started
backup_completeBackup operation completed
backup_failedBackup operation failed
stats_anomalyModule restarted, went stale, or connector lost/gained channels

Some events are suppressed from the stdout renderer (first-cycle discovery, passive anomaly detection, stale module detection) — they’re still processed for state tracking, but not printed as event lines. Raw mode (--raw) shows everything.

NATS Subjects

Inbound (subscribed)

SubjectPurpose
stats.emit.>Passive anomaly detection between summary cycles
control.connectors.>Connector lifecycle events
command.registerCommand registration notifications
command.unregisterCommand unregistration notifications
broadcast.registerBroadcast registration notifications
broadcast.unregisterBroadcast unregistration notifications
help.updateHelp entry updates
help.removeHelp entry removals
control.registerCommands.>Router command re-registration sweeps
control.registerBroadcasts.>Router broadcast re-registration sweeps
stats.emit.requestStats collection requests (monitor responds with its own stats)
stats.uptimeUptime queries

Outbound (published)

SubjectPurpose
stats.emit.requestFan-out to collect stats from all modules (with reply channel)
stats.emit.response.<uuid>Reply channel for stats collection responses

Configuration

To deploy the CLI module, add it to your bot’s botModules configuration with moduleName: "cli":

  botModules:
- name: cli
  spec:
    size: 1
    image: ghcr.io/eeveebot/cli:latest
    pullPolicy: Always
    ipcConfig: my-eevee-bot
    moduleName: cli
    moduleConfig: |
      monitor:
        summaryInterval: 60000
        maxModuleAge: 300000
        displayEvents:
          - module_start
          - module_error
          - connector_disconnect
          - stats_anomaly
        filters:
          - chat.irc
  

Configuration Fields

FieldTypeDefaultDescription
summaryIntervalnumber60000Milliseconds between summary table refreshes
maxModuleAgenumber300000Milliseconds before a module with no stats is considered stale
displayEventsstring[](all)Event types shown in follow mode
filtersstring[](none)Subject prefix filters

Environment Variables

VariableRequiredDefaultDescription
NATS_HOSTYesNATS server URL
NATS_TOKENYesNATS authentication token
MODULE_CONFIG_PATHNoPath to YAML config file (falls back to defaults if unset)
HTTP_API_PORTNo9000Port for metrics and health-check HTTP server

Toolbox Container

The toolbox image bundles eevee monitor as its default entrypoint. On startup, it:

  1. Runs init hooks from /eevee/hook.d/init/ (prints hostname and IP for debugging)
  2. Launches eevee monitor in follow mode
  docker run --rm \
  -e NATS_HOST="nats://nats.example.com:4222" \
  -e NATS_TOKEN="my-secret-token" \
  ghcr.io/eeveebot/cli:latest
  

The image also includes the nats CLI for manual NATS inspection:

  docker exec -it <container> bash
nats sub ">"
  

Anomaly Detection

The monitor detects anomalies by comparing current and previous module state:

  • Uptime reset — module restarted (current uptime < previous uptime)
  • Error count increase — any delta triggers a module_error event
  • Stale module — no stats received within maxModuleAge → module marked down
  • Status recovery — module transitioned from down/degraded to healthy
  • Channel count change — connector gained or lost channels

Connector data is extracted from connector module stats responses (e.g. connector-irc includes a connector array in its stats payload).

Monitor Stats

The monitor reports its own stats when queried via stats.emit.request:

FieldDescription
modules_observedTotal modules in state table
modules_healthyModules with status healthy
modules_degradedModules with status degraded
modules_downModules with status down or stale
connectors_observedTotal connectors in state table
connectors_connectedConnectors with status connected
events_processedTotal events emitted to renderer
summary_intervals_completedNumber of summary cycles completed