Differences

This shows you the differences between two versions of the page.

--- infra:operations:proxmox-maintenance [2026/05/31 19:41] – Document completed Proxmox follow-up actions atluxity_idp.hackeriet.no
+++ infra:operations:proxmox-maintenance [2026/06/01 17:32] (current) – Note LibreNMS alerting follow-up for klynge001 atluxity_idp.hackeriet.no
@@ Line 152: / Line 152: @@
   * ''nc -vz -w3 <ip> 22'' means SSH is listening, if SSH is expected for that guest.
   * Lack of ping or SSH does not always mean the guest is broken. Investigate only failures that contradict expectations for that guest.
+===== Monitoring / LibreNMS =====
+As of 2026-06-01, host006 and host007 are monitored in LibreNMS as Proxmox hypervisors. This replaces the old Munin host monitoring for these nodes.
+LibreNMS records:
+  * host006.hackeriet.no: LibreNMS device ID 25.
+  * host007.hackeriet.no: LibreNMS device ID 26.
+  * Both devices have OS detected as ''proxmox''.
+  * Both devices have the LibreNMS ''proxmox'' application enabled with app instance ''klynge001''.
+Host-side setup:
+  * ''snmpd'' is installed, enabled, and running on both hosts.
+  * SNMP listens only on the management IP of each host:
+    * host006: ''10.10.50.26:161/udp''
+    * host007: ''10.10.50.27:161/udp''
+  * SNMP uses SNMPv3 authPriv with the security name ''librenms_klynge001''.
+  * Credentials are stored in Hackerpass at ''infrastructure/librenms-klynge001-snmpv3''. Do not put the credential values in the wiki or NetBox.
+  * The Proxmox LibreNMS agent-local script is installed as ''/usr/local/libexec/librenms-proxmox''.
+  * SNMP exposes Proxmox VM traffic through:
+<code>
+extend proxmox /usr/bin/sudo /usr/local/libexec/librenms-proxmox
+</code>
+  * ''/etc/sudoers.d/librenms-proxmox'' allows the ''Debian-snmp'' user to run only that script via sudo.
+Firewall setup:
+  * The Proxmox cluster firewall allows SNMP only from app-01 / LibreNMS source address ''10.10.50.51'' to the ''klynge001'' IP set.
+  * The firewall rule is in ''/etc/pve/firewall/cluster.fw'':
+<code>
+IN ACCEPT -source 10.10.50.51/32 -dest +klynge001 -p udp -dport 161 -log nolog # LibreNMS SNMP polling from app-01
+</code>
+LibreNMS setup:
+  * ''enable_proxmox'' is set to ''true''.
+  * The ''unix-agent'' poller module is disabled per device for host006 and host007, because this setup uses SNMP extend instead of the LibreNMS unix-agent on port 6556.
+  * No LibreNMS alert rules existed when this monitoring was added. Host metrics and Proxmox application polling are live, but alert policy still needs to be defined.
+Useful verification commands:
+<code>
+# On each Proxmox host
+systemctl is-active snmpd
+ss -lunp | grep ':161'
+sudo -u Debian-snmp sudo /usr/local/libexec/librenms-proxmox
+# From the LibreNMS container on app-01
+snmpwalk -v3 -l authPriv -u librenms_klynge001 -a SHA -A '<auth password>' -x AES -X '<privacy password>' host006.hackeriet.no SNMPv2-MIB::sysName.0
+snmpget -v3 -l authPriv -u librenms_klynge001 -a SHA -A '<auth password>' -x AES -X '<privacy password>' -Oqv host006.hackeriet.no .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.112.114.111.120.109.111.120
+lnms device:poll -m applications host006.hackeriet.no
+</code>
 ===== Follow-up items =====
+  * Define LibreNMS alert rules and notification routing for klynge001 host monitoring.
   * Investigate the host007 GRUB/LVM ''pv1'' warning.
   * Investigate the host006 EFI fallback warning.