Bug 1624879

Summary: The libvirtMib_subagent sends SNMP traps without sysUptime.0
Product: Red Hat Enterprise Linux 7 Reporter: Miguel Martin <mmartinv>
Component: libvirt-snmpAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: jiyan <jiyan>
Severity: medium Docs Contact:
Priority: high    
Version: 7.5CC: dyuan, gveitmic, jherrman, jiyan, jsafrane, lmen, mmartinv, mprivozn, mtessun, xuzhang, yalzhang
Target Milestone: rcKeywords: Upstream, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-snmp-0.0.3-6.el7 Doc Type: Bug Fix
Doc Text:
Prior to this update, the variables sent in the SNMP trap by libvirt-snmp did not conform to the RFC 1905 protocol operations standard. This update fixes the problem.
 Story Points: ---
Clone Of:
: 1653591 (view as bug list) Environment:
Last Closed: 2019-08-06 12:01:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1653591    

Description Miguel Martin 2018-09-03 13:33:00 UTC 
Description of problem:

libvirtMib_subagent does not conform RFC1905:
According to [1]:
"The first two variable bindings in the variable binding list of an
 SNMPv2-Trap-PDU are sysUpTime.0 [9] and snmpTrapOID.0 [9]
 respectively."

snmpTrapOID.0 is the first variable binding in src/libvirtNotifications.c code and no sysUptime.0 is sent 

Version-Release number of selected component (if applicable):
libvirt-snmp-0.0.3

Additional info:
[1] https://tools.ietf.org/html/rfc1905#section-4.2.6

A possible patch would be:

diff --git a/src/libvirtNotifications.c b/src/libvirtNotifications.c
index 135f177..29419b7 100644
--- a/src/libvirtNotifications.c
+++ b/src/libvirtNotifications.c
@@ -28,6 +28,7 @@
 #include "libvirtGuestTable_enums.h"
 #include "libvirtSnmpError.h"
 
+static const oid sysuptime_oid[] = { 1, 3, 6, 2, 1, 1, 3, 0 };
 static const oid snmptrap_oid[] = { 1, 3, 6, 1, 6, 3, 1, 1, 4, 1, 0 };
 
 int
@@ -49,6 +50,7 @@ send_libvirtGuestNotif_trap(virDomainPtr dom)
     unsigned char domUUID[VIR_UUID_BUFLEN];
     virDomainInfo info;
     int rowstatus = ROWSTATUS_ACTIVE;
+    u_long uptime = netsnmp_get_agent_uptime();
 
     if (virDomainGetUUID(dom, domUUID) < 0) {
         printLibvirtError("Failed to get domain UUID");
@@ -76,6 +78,15 @@ send_libvirtGuestNotif_trap(virDomainPtr dom)
             break;
     };
 
+    /*
+     * Set the sysUptime.0 value
+     */
+    snmp_varlist_add_variable(&var_list,
+                              sysuptime_oid, OID_LENGTH(sysuptime_oid),
+                              ASN_TIMETICKS,
+                              (u_char*)&uptime,
+                              sizeof(uptime));
+
     /*
      * Set the snmpTrapOid.0 value
      */
Comment 2 Michal Privoznik 2018-09-10 13:41:49 UTC 
Pushed upstream:

commit 0ca48c8abe6e29ee0a61f95f67cfc1da0e486b70
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Sep 10 15:36:05 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Mon Sep 10 15:36:05 2018 +0200

    Send sysUpTime in traps
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1624879
    
    According to RFC 1905:
    
      The first two variable bindings in the variable binding list of
      an SNMPv2-Trap-PDU are sysUpTime.0 and snmpTrapOID.0
      respectively.
    
    We are setting the snmpTrapOID.0 variable but not sysUpTime.0.
    
    Based on work of: Miguel Martin <mmartinv>
    
    Signed-off-by: Michal Privoznik <mprivozn>

v0.0.3-15-g0ca48c8
Comment 3 jiyan 2018-10-17 05:54:57 UTC 
Verifying steps can be checked in https://bugzilla.redhat.com/show_bug.cgi?id=1624839#c7
Comment 4 Michal Privoznik 2018-10-23 10:28:33 UTC 
Moving to POST per comment 2.
Comment 6 jiyan 2018-11-12 11:00:07 UTC 
Configuration:
1. Install libvirt-snmp, libvirt-snmp-debuginfo, net-snmp net-snmp-libs, net-snmp-utils components
2. Modify "/etc/snmp/snmpd.conf" as following: 
   # cat /etc/snmp/snmpd.conf
   rwcommunity public
   master agentx
   trapcommunity public
   trap2sink localhost
3. Modify "/etc/snmp/snmptrapd.conf" as following: 
   # cat /etc/snmp/snmptrapd.conf 
   authCommunity log,execute,net public 
   logOption f /var/log/snmptraps.log
4. Modify "/etc/sysconfig/snmptrapd" as following: 
   # cat /etc/sysconfig/snmptrapd 
   OPTIONS="-m ALL -p /var/run/snmptrapd.pid"
5. Restart related service
   # systemctl restart snmpd
   # systemctl restart snmptrapd
   # systemctl restart libvirtd

**Host1:**
Version:
**********libvirt-snmp-0.0.3-6.el7.x86_64
kernel-3.10.0-957.el7.x86_64
qemu-kvm-rhev-2.12.0-19.el7_6.2.x86_64
libvirt-4.5.0-10.virtcov.el7_6.3.x86_64

Steps:
1. Run libvirt snmp agent daemon through the following cmd on "Terminal 1"
# LIBVIRT_DEFAULT_URI="qemu:///system" libvirtMib_subagent -f -L -D ALL

2. Run snmpwalk to query domain status on "Terminal 2"
# snmpwalk -m ALL -v 2c -c public -OX localhost libvirtMIB
LIBVIRT-MIB::libvirtGuestName[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = STRING: "test1"
LIBVIRT-MIB::libvirtGuestState[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = INTEGER: running(1)
LIBVIRT-MIB::libvirtGuestCpuCount[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = Gauge32: 1
LIBVIRT-MIB::libvirtGuestMemoryCurrent[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = Gauge32: 1024
LIBVIRT-MIB::libvirtGuestMemoryLimit[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = Gauge32: 1024
LIBVIRT-MIB::libvirtGuestCpuTime[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = Counter64: 19100000000
LIBVIRT-MIB::libvirtGuestRowStatus[STRING: 15966392-65dc-4306-9dfe-3bd92e641436] = INTEGER: active(1)

which is identical with the output of 'virsh' cmd:
# virsh domstate test1
running

3. Open ths third terminal to check the log
# tail -f  /var/log/messages | grep --line-buffered "localhost [UDP: [127.0.0.1]"

4. Set the status of the domain through 'snmpset' cmd
# snmpset -m ALL -v 2c -c public localhost libvirtGuestState.\'15966392-65dc-4306-9dfe-3bd92e641436\' = paused
LIBVIRT-MIB::libvirtGuestState.'..c.e.C...;..d.6' = INTEGER: paused(3)

Verify the status of domain through 'virsh' cmd:
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 3     test1                          paused

5. Check the log
# tail -f  /var/log/messages | grep --line-buffered "localhost [UDP: [127.0.0.1]"
Nov 12 05:50:22 ibm-x3250m6-05 snmptrapd[16211]: localhost [UDP: [127.0.0.1]:37508->[127.0.0.1]:162]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (6733) 0:01:07.33, SNMPv2-MIB::snmpTrapOID.0 = OID: LIBVIRT-MIB::libvirtGuestNotif, LIBVIRT-MIB::libvirtGuestName.0 = STRING: "test1", LIBVIRT-MIB::libvirtGuestUUID.0 = STRING: 15966392-65dc-4306-9dfe-3bd92e641436, LIBVIRT-MIB::libvirtGuestState.0 = INTEGER: paused(3), LIBVIRT-MIB::libvirtGuestRowStatus.0 = INTEGER: active(1)
Nov 12 05:50:22 ibm-x3250m6-05 snmptrapd[16211]: localhost [UDP: [127.0.0.1]:37508->[127.0.0.1]:162]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (6733) 0:01:07.33, SNMPv2-MIB::snmpTrapOID.0 = OID: LIBVIRT-MIB::libvirtGuestNotif, LIBVIRT-MIB::libvirtGuestName.0 = STRING: "test1", LIBVIRT-MIB::libvirtGuestUUID.0 = STRING: 15966392-65dc-4306-9dfe-3bd92e641436, LIBVIRT-MIB::libvirtGuestState.0 = INTEGER: paused(3), LIBVIRT-MIB::libvirtGuestRowStatus.0 = INTEGER: active(1)


**Host2:**
Version:
**********libvirt-snmp-0.0.3-5.el7.x86_64
libvirt-4.5.0-10.virtcov.el7_6.3.x86_64
qemu-kvm-rhev-2.12.0-19.el7_6.2.x86_64
kernel-3.10.0-957.el7.x86_64

Steps:
1. Run libvirt snmp agent daemon through the following cmd on "Terminal 1"
# LIBVIRT_DEFAULT_URI="qemu:///system" libvirtMib_subagent -f -L -D ALL

2. Run snmpwalk to query domain status on "Terminal 2"
# snmpwalk -m ALL -v 2c -c public -OX localhost libvirtMIB
LIBVIRT-MIB::libvirtGuestName[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = STRING: "test1"
LIBVIRT-MIB::libvirtGuestState[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = INTEGER: running(1)
LIBVIRT-MIB::libvirtGuestCpuCount[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = Gauge32: 1
LIBVIRT-MIB::libvirtGuestMemoryCurrent[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = Gauge32: 1024
LIBVIRT-MIB::libvirtGuestMemoryLimit[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = Gauge32: 1024
LIBVIRT-MIB::libvirtGuestCpuTime[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = Counter64: 7430000000
LIBVIRT-MIB::libvirtGuestRowStatus[STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360] = INTEGER: active(1)

which is identical with the output of 'virsh' cmd:
# virsh domstate test1
running

3. Open ths third terminal to check the log
# tail -f  /var/log/messages | grep --line-buffered "localhost [UDP: [127.0.0.1]"

4. Set the status of the domain through 'snmpset' cmd
# snmpset -m ALL -v 2c -c public localhost libvirtGuestState.\'20ae5bfa-a34c-4246-8f3e-f80096771360\' = paused
LIBVIRT-MIB::libvirtGuestState.' .[..LBF.>...w.`' = INTEGER: paused(3)

Verify the status of domain through 'virsh' cmd:
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 3     test1                          paused

5. Check the log
# tail -f  /var/log/messages | grep --line-buffered "localhost [UDP: [127.0.0.1]"
Nov 12 05:52:42 ibm-x3850x6-03 snmptrapd[24582]: localhost [UDP: [127.0.0.1]:33611->[127.0.0.1]:162]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (8766) 0:01:27.66, SNMPv2-MIB::snmpTrapOID.0 = OID: LIBVIRT-MIB::libvirtGuestNotif, LIBVIRT-MIB::libvirtGuestName.0 = STRING: "test1", LIBVIRT-MIB::libvirtGuestUUID.1 = STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360, LIBVIRT-MIB::libvirtGuestState.2 = INTEGER: paused(3), LIBVIRT-MIB::libvirtGuestRowStatus.3 = INTEGER: active(1)
Nov 12 05:52:42 ibm-x3850x6-03 snmptrapd[24582]: localhost [UDP: [127.0.0.1]:33611->[127.0.0.1]:162]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (8767) 0:01:27.67, SNMPv2-MIB::snmpTrapOID.0 = OID: LIBVIRT-MIB::libvirtGuestNotif, LIBVIRT-MIB::libvirtGuestName.0 = STRING: "test1", LIBVIRT-MIB::libvirtGuestUUID.1 = STRING: 20ae5bfa-a34c-4246-8f3e-f80096771360, LIBVIRT-MIB::libvirtGuestState.2 = INTEGER: paused(3), LIBVIRT-MIB::libvirtGuestRowStatus.3 = INTEGER: active(1)

Hi Michal.
As we have discussed several days ago, in both old version and new version of 'libvirt-snmp', I can see the 'sysUpTimeInstance' info through the log.

But in the new version, the code about the 'sysuptime' is truly added, so I can not distinguish whether 'sysUpTimeInstance' info is generated by the new code.
Comment 7 Michal Privoznik 2018-11-12 13:25:25 UTC 
(In reply to jiyan from comment #6)

> Hi Michal.
> As we have discussed several days ago, in both old version and new version
> of 'libvirt-snmp', I can see the 'sysUpTimeInstance' info through the log.
> 
> But in the new version, the code about the 'sysuptime' is truly added, so I
> can not distinguish whether 'sysUpTimeInstance' info is generated by the new
> code.

What we could do is run the agent in stand alone mode and see if it sends the instance uptime. Or maybe previously it was net-snmp code adding it if not found? Jan, you originally wrote this code, any ideas please?
Comment 8 Miguel Martin 2018-11-26 08:32:44 UTC 
It looks like the net-snmp code prepends the sysUptime.0 varbind if not present:
~~~
       /*
         * Check the varbind list we've been given.
         * If it starts with a 'sysUptime.0' varbind, then use that.
         * Otherwise, prepend a suitable 'sysUptime.0' varbind.
         */
        if (!snmp_oid_compare( vblist->name,    vblist->name_length,
                               sysuptime_oid, sysuptime_oid_len )) {
            template_v2pdu->variables = vblist;
            trap_vb  = vblist->next_variable;
        } else {
            uptime   = netsnmp_get_agent_uptime();
            var = NULL;
            snmp_varlist_add_variable( &var,
                           sysuptime_oid, sysuptime_oid_len,
                           ASN_TIMETICKS, (u_char*)&uptime, sizeof(uptime));
            if (!var) {
                snmp_log(LOG_WARNING,
                     "send_trap: failed to insert sysUptime varbind\n");
                snmp_free_pdu(template_v2pdu);
                snmp_free_varbind(vblist);
                return -1;
            }
            template_v2pdu->variables = var;
            var->next_variable        = vblist;
            trap_vb  = vblist;
        }
~~~

So it seems the patch is not necessary. :(
Comment 10 jiyan 2018-11-28 07:05:19 UTC 
Hi Miguel
So how should we deal with this bug? Should we close this bug OR other solutions?
Comment 12 Miguel Martin 2018-12-03 17:08:27 UTC 
(In reply to jiyan from comment #10)
> Hi Miguel
> So how should we deal with this bug? Should we close this bug OR other
> solutions?

Not sure TBH, the bug has all the flags granted and it's not harmful although is not needed.

Michal what do you think?
Comment 13 Michal Privoznik 2018-12-04 07:39:33 UTC 
(In reply to Miguel Martin from comment #12)
> (In reply to jiyan from comment #10)
> > Hi Miguel
> > So how should we deal with this bug? Should we close this bug OR other
> > solutions?
> 
> Not sure TBH, the bug has all the flags granted and it's not harmful
> although is not needed.
> 
> Michal what do you think?

Yeah, I think it makes sense to have the patch in. Moreover, if there is a way to run agentx in a standalone mode (it looks like it should be possible but I'm unable to come up with some usable config file for it), then the trapd doesn't add sysUptime right? In that case the agent should do that itself.
Comment 14 jiyan 2018-12-04 07:57:20 UTC 
Hi Michal
So Do you think whether the steps in comment 6 are enough to verify this bug?
Or should I do something else?

https://bugzilla.redhat.com/show_bug.cgi?id=1624879#c6
Comment 15 Michal Privoznik 2018-12-04 09:42:01 UTC 
(In reply to jiyan from comment #14)
> Hi Michal
> So Do you think whether the steps in comment 6 are enough to verify this bug?
> Or should I do something else?

Yes, they are enough. Thanks.
Comment 16 jiyan 2018-12-05 01:56:19 UTC 
According to comment 15, move this bug to be verified.
Comment 18 errata-xmlrpc 2019-08-06 12:01:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2013