snmp_exporter for the CARC PLG stack (vortex)

Deploys prom/snmp-exporter on vortex.alliance.unm.edu as a rootful Podman Quadlet unit, polling the Mitsubishi UPS at 129.24.240.37 over SNMPv2c using the standard UPS-MIB (RFC 1628), and wires a scrape job into the existing /etc/monitoring/prometheus/prometheus.yml.

Layout

ansible.cfg                 # inventory path, become prompts for sudo password
inventory.ini                # [monitoring] group -> vortex
site.yml                     # entry-point playbook
group_vars/monitoring.yml    # non-secret vars (UPS IP, vendor)
roles/snmp_exporter/
  defaults/main.yml          # image version, ports, paths — check/bump snmp_exporter_version
  tasks/main.yml             # deploy config, Quadlet unit, enable service, patch prometheus.yml
  handlers/main.yml          # restart snmp_exporter / restart prometheus container
  templates/
    snmp.yml.j2              # hand-written UPS-MIB module (not generator-produced)
    snmp_exporter.container.j2   # Podman Quadlet unit
    prometheus_snmp_job.yml.j2   # scrape job block inserted into prometheus.yml

What it does

1. Writes /etc/monitoring/snmp_exporter/snmp.yml (mode 0640, root-only — it contains the community string).
2. Writes /etc/containers/systemd/snmp_exporter.container (Quadlet unit), daemon-reloads, enables + starts snmp_exporter.service.
   • Runs with Network=host, so it listens on the host's :9116 directly — no podman network coupling with the other containers needed.
3. Inserts a marked, idempotent block into prometheus.yml's scrape_configs: (via blockinfile, safe to re-run) targeting vortex.alliance.unm.edu:9116 with module: [ups].
4. Runs promtool check config inside the running prometheus container before reloading — if the config is invalid, the play aborts and does not restart Prometheus.
5. Restarts the prometheus container (podman restart prometheus) to pick up the new job.

Before you run it

• Bump the pin. roles/snmp_exporter/defaults/main.yml pins snmp_exporter_version: v0.27.0. Check https://github.com/prometheus/snmp_exporter/releases and update if a newer release exists.
• Verify the promtool path. prometheus_container_config_path in defaults assumes the prometheus container mounts its config at the image's default /etc/prometheus/prometheus.yml. Confirm with sudo podman inspect prometheus | grep -A2 Mounts and adjust the var (via -e or group_vars/monitoring.yml) if it differs.
• Firewall. Since snmp_exporter uses host networking, if firewalld is active on vortex and the prometheus container reaches the host over an interface other than loopback, you may need to allow tcp/9116 from the podman bridge subnet. Not handled by this playbook — check sudo firewall-cmd --list-all first.
• You need the UPS's actual SNMPv2c community string. It is deliberately not stored anywhere in this repo.

Running it

External SSH to vortex is blocked, so this runs locally on the box itself (ansible_connection=local in inventory.ini) rather than from an external control machine. Get the repo onto vortex (e.g. git clone or scp -r from wherever you have campus-network/console access), then from a shell on vortex:

ansible-playbook site.yml -e snmp_ups_community='<the real community string>'

You'll be prompted for the local sudo (become) password (become_ask_pass = True in ansible.cfg).

If snmp_ups_community is omitted, the snmp.yml template fails fast with a clear error instead of writing a broken/empty community string.

Verifying

Run these directly on vortex:

sudo systemctl status snmp_exporter.service
curl -s http://localhost:9116/snmp?target=129.24.240.37\&module=ups | grep ^ups | head -30
sudo podman logs prometheus --tail 50
# in a browser: http://vortex.alliance.unm.edu (via nginx) -> Prometheus -> Status -> Targets -> snmp_ups

Extending later (switches, PDUs)

• Add a new modules: entry to snmp.yml.j2 (e.g. if_mib for IF-MIB-based switches, or a vendor MIB for PDUs/environmental sensors).
• Add a matching job_name block to a new template (or extend prometheus_snmp_job.yml.j2) with the right module: param and target list.
• Since v0.23.0, snmp_exporter splits auth out of the module: define a new named profile under the top-level auths: map (e.g. snmpv3_switch) with version: 3, username, security_level, auth_protocol, password, priv_protocol, priv_password — pull those from new -e extra-vars the same way snmp_ups_community is handled now, don't hardcode them — and reference it via params: { auth: [snmpv3_switch] } in that module's scrape job (see public_v2 in prometheus_snmp_job.yml.j2 for the pattern). See https://github.com/prometheus/snmp_exporter/blob/main/auth-split-migration.md for background.