--- Additional comment from on 2019-09-17 22:14:43 CEST --- fedora 30, snmp 5.8-10 ==11498== Invalid free() / delete / delete[] / realloc() ==11498== at 0x4839A0C: free (vg_replace_malloc.c:540) ==11498== by 0x4C548CC: usm_rgenerate_out_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C55458: usm_secmod_rgenerate_out_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF46B8: snmpv3_packet_realloc_rbuild (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF86B4: snmp_build (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF8DC9: netsnmp_build_packet (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF9026: _build_initial_pdu_packet (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BFAE8A: snmp_sess_async_send (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x48A5817: netsnmp_wrap_up_request (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x48A82AD: netsnmp_check_delegated_requests (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x48A8DF0: netsnmp_check_outstanding_agent_requests (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x10D948: ??? (in /usr/sbin/snmpd) ==11498== Address 0x14491000 is 0 bytes inside a block of size 104 free'd ==11498== at 0x4839A0C: free (vg_replace_malloc.c:540) ==11498== by 0x4C53A57: usm_rgenerate_out_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C55458: usm_secmod_rgenerate_out_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF46B8: snmpv3_packet_realloc_rbuild (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF86B4: snmp_build (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF8DC9: netsnmp_build_packet (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BF9026: _build_initial_pdu_packet (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x48A5747: netsnmp_wrap_up_request (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x48A82AD: netsnmp_check_delegated_requests (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x48A8DF0: netsnmp_check_outstanding_agent_requests (in /usr/lib64/libnetsnmpagent.so.35.0.0) ==11498== by 0x10D948: ??? (in /usr/sbin/snmpd) ==11498== by 0x10D1D4: ??? (in /usr/sbin/snmpd) ==11498== Block was alloc'd at ==11498== at 0x483AB1A: calloc (vg_replace_malloc.c:762) ==11498== by 0x4C50B00: usm_process_in_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C51E8C: usm_secmod_process_in_msg (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BFEC70: snmpv3_parse (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4BFFF72: ??? (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C01846: ??? (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C02115: _sess_read (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C02C3C: snmp_sess_read2 (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x4C02C8A: snmp_read2 (in /usr/lib64/libnetsnmp.so.35.0.0) ==11498== by 0x10DD81: ??? (in /usr/sbin/snmpd) ==11498== by 0x10D1D4: ??? (in /usr/sbin/snmpd) ==11498== by 0x4D9FF42: (below main) (in /usr/lib64/libc-2.29.so) --- Additional comment from Josef Ridky on 2019-09-18 08:43:08 CEST --- @Pavel do you have trap forwarding enabled in configuration? --- Additional comment from on 2019-09-18 11:47:06 CEST --- No, it is disabled. In my config snmpd is master (agentx), agentx transport is unix sockets and subagent is agentx client based on agent++ library. security model is USM. interesting it fails only on some tables (and no problem with the same table in <= fc28 os) under valgrind it does not crash (think because of valgrind virtual machine) and shows some interesting messages like send response: USM encryption error (build string: bad header, length too short: 2 < 11) or send response: USM encryption error or send response: USM encryption error (Can't build OID for variable) i tried to review subagent source code but didnt manage to find any difference between "good" and "bad" tables may be PDU size limitation ? "bad" table contains 7 columns and one of columns has length ~ 200 characters, snmpd does not crash when subagent returns 1 table entry but immediately crashes when i add second record. anyway i don't see this crash in previous versions of snmpd
I have prepared scratch build with possible solution. @pavel Are you able to test it on your case? Scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=37736050
5.8-13, fc30: snmpd daemon do not crash, command line tools work as required but now i see problem with perl module SNMP::Session i have test table with 2000 records, snmptable fetches all rows, SNMP::Session::get_table stops after ~127 rows with status: (genError) A general failure occured (tooBig) Response message would have been too large. can successfully retrieve the same table under fc28
This issue also appears to affect RHEL/CentOS 8. The snmpd service crashes every few hours with log entries similar to below. Dec 30 01:25:39 host.example.com systemd-coredump[788]: Process 30476 (snmpd) of user 0 dumped core. Stack trace of thread 30476: #0 0x00007f26c126e93f raise (libc.so.6) #1 0x00007f26c1258c95 abort (libc.so.6) #2 0x00007f26c12b1d57 __libc_message (libc.so.6) #3 0x00007f26c12b868c malloc_printerr (libc.so.6) #4 0x00007f26c12ba027 _int_free (libc.so.6) #5 0x00007f26c2bd1275 usm_generate_out_msg (libnetsnmp.so.35) #6 0x00007f26c2bd1ae9 usm_secmod_generate_out_msg (libnetsnmp.so.35) #7 0x00007f26c2b74ed2 snmpv3_packet_build (libnetsnmp.so.35) #8 0x00007f26c2b76f07 snmp_build (libnetsnmp.so.35) #9 0x00007f26c2b7742b netsnmp_build_packet (libnetsnmp.so.35) #10 0x00007f26c2b776d6 _build_initial_pdu_packet (libnetsnmp.so.35) #11 0x00007f26c33df5c9 netsnmp_wrap_up_request (libnetsnmpagent.so.35) #12 0x00007f26c33e277b netsnmp_handle_request (libnetsnmpagent.so.35) #13 0x00007f26c33e2a9a handle_snmp_packet (libnetsnmpagent.so.35) #14 0x00007f26c2b7f712 _sess_process_packet (libnetsnmp.so.35) #15 0x00007f26c2b80776 _sess_read (libnetsnmp.so.35) #16 0x00007f26c2b812dd snmp_sess_read2 (libnetsnmp.so.35) #17 0x00007f26c2b8132b snmp_read2 (libnetsnmp.so.35) #18 0x000055dd2afb9377 receive (snmpd) #19 0x000055dd2afb866e main (snmpd) #20 0x00007f26c125a813 __libc_start_main (libc.so.6) #21 0x000055dd2afb8abe _start (snmpd) Package info is as follows. [root@host ~]# rpm -qi net-snmp Name : net-snmp Epoch : 1 Version : 5.8 Release : 7.el8_0.2 Architecture: x86_64
FEDORA-2020-c3ab547325 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-c3ab547325
FEDORA-2020-44b769adaf has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-44b769adaf
FEDORA-2020-7303a69282 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-7303a69282
FEDORA-2020-c3ab547325 has been pushed to the Fedora 31 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-c3ab547325` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-c3ab547325 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2020-44b769adaf has been pushed to the Fedora 30 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-44b769adaf` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-44b769adaf See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2020-7303a69282 has been pushed to the Fedora 32 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-7303a69282` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-7303a69282 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2020-44b769adaf has been pushed to the Fedora 30 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2020-7303a69282 has been pushed to the Fedora 32 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2020-c3ab547325 has been pushed to the Fedora 31 stable repository. If problem still persists, please make note of it in this bug report.