Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1823841

Summary: Double free in netsnmp_handler_free when snmpd exits
Product: Red Hat Enterprise Linux 7 Reporter: Renaud Métrich <rmetrich>
Component: net-snmpAssignee: Josef Ridky <jridky>
Status: CLOSED WONTFIX QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.7CC: fkrska, jridky, myamazak, sbroz
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1875698 (view as bug list) Environment:
Last Closed: 2020-09-04 05:50:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1875698    

Description Renaud Métrich 2020-04-14 15:28:27 UTC
Description of problem:

When using an extend, a double-free occurs when snmpd service shuts down.


Version-Release number of selected component (if applicable):

net-snmp-5.7.2-48.el7_8.x86_64


How reproducible:

ALWAYS

Steps to Reproduce:

1. Patch /etc/snmp/snmpd.conf as shown below

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
-view    systemview    included   .1.3.6.1.2.1.1
-view    systemview    included   .1.3.6.1.2.1.25.1.1
+#view    systemview    included   .1.3.6.1.2.1.1
+#view    systemview    included   .1.3.6.1.2.1.25.1.1
+view    all    included    .1 80

...

-access  notConfigGroup ""      any       noauth    exact  systemview none none
+#access  notConfigGroup ""      any       noauth    exact  systemview none none
+access  notConfigGroup ""      any       noauth    exact  all all none

...

+extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

2. Start snmpd

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
# /usr/sbin/snmpd  -f -LS0-6d
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

3. Execute a snmpwalk command triggering mpstat (can be installed, or not, same result)

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.8
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

4. Stop snmpd

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
# pkill -TERM snmpd
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------


Actual results:

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
*** Error in `/usr/sbin/snmpd': free(): invalid pointer: 0xXXX ***
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Additional info:

The backtrace shows it crashes while freeing a pointer. For sure there is a double-free but so far I couldn't spot who freed first:

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
 761 void
 762 netsnmp_handler_registration_free(netsnmp_handler_registration *reginfo)
 763 {
 764     if (reginfo != NULL) {
 765         netsnmp_handler_free(reginfo->handler);	<---- HERE
 766         SNMP_FREE(reginfo->handlerName);
 767         SNMP_FREE(reginfo->contextName);
 768         SNMP_FREE(reginfo->rootoid);
 769         reginfo->rootoid_len = 0;
 770         SNMP_FREE(reginfo);
 771     }
 772 }
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Comment 2 Renaud Métrich 2020-04-15 08:52:48 UTC
It's unclear to me why this doesn't show up under Valgrind, which would be very helpful finding the exact place where the first free() happened.

Anyway, I now have a better view of what happens.
During shutdown, we have the following:

1. Extend gets unregistered, which frees the "ereg_head" global list

gdb) bt
#0  _unregister_extend (eptr=0x555555883a50) at agent/extend.c:260
#1  shutdown_extend () at agent/extend.c:316
#2  0x00007ffff77d0915 in _shutdown_mib_modules (majorID=<optimized out>, minorID=<optimized out>, 
    serve=<optimized out>, client=<optimized out>) at ../agent/mibgroup/mib_module_shutdown.h:34
#3  0x00007ffff721354f in snmp_call_callbacks (major=major@entry=0, minor=minor@entry=2, 
    caller_arg=caller_arg@entry=0x0) at callback.c:363
#4  0x00007ffff71e62f7 in snmp_shutdown (type=<optimized out>) at snmp_api.c:910
#5  0x00005555555579d4 in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1135

2. Cache tree gets freed, leading again to freeing the already freed extend

(gdb) bt
#0  0x00007ffff574d387 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007ffff574ea78 in __GI_abort () at abort.c:90
#2  0x00007ffff578fed7 in __libc_message (do_abort=do_abort@entry=2, 
    fmt=fmt@entry=0x7ffff58a2350 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
#3  0x00007ffff5798299 in malloc_printerr (ar_ptr=0x7ffff5ade760 <main_arena>, ptr=<optimized out>, 
    str=0x7ffff589fb50 "free(): invalid pointer", action=3) at malloc.c:4967
#4  _int_free (av=0x7ffff5ade760 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:3843
#5  0x00007ffff7b95b02 in netsnmp_handler_registration_free (reginfo=0x5555559ba7b0)
    at agent_handler.c:765
#6  0x00007ffff7b9886e in netsnmp_subtree_free (a=0x5555559ba6c0) at agent_registry.c:474
#7  0x00007ffff7b9915c in clear_subtree (sub=<optimized out>) at agent_registry.c:961
#8  0x00007ffff7b991ac in clear_context () at agent_registry.c:427
#9  0x00007ffff7ba4f1e in shutdown_agent () at snmp_vars.c:370
#10 0x00005555555579de in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1137

In frame 6, we have the subtree reference the extend handler (reginfo), which has already been freed in step 1, causing the crash:

 21 typedef struct netsnmp_subtree_s {
 :
 41     netsnmp_handler_registration *reginfo;      /* new API */
 :
 45 } netsnmp_subtree;


This is quite expected, since after unregistering the extend, the "reginfo" in the subtree in now a dangling pointer.

Comment 4 Renaud Métrich 2020-04-16 09:49:21 UTC
Another scenario is while performing a reload:

1. Have the extend in /etc/snmp/snmpd.conf

    extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL

2. Start snmpd

    # /usr/sbin/snmpd -f -Le0-6d

3. Remove the extend

    #extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL

4. Reload snmpd

    # pkill -HUP snmpd


Oddly, running under valgrind prevents the crash from happening at all.