Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
Customer applied updates on systemd in Rhel 8.4 EUS but was still affected by a bug.
Systemd still crashes in version systemd-239-45.el8_4.6.x86_64 generating coredump.
It should be solved by Bugzilla rhbz#2026244 per errata : https://access.redhat.com/errata/RHBA-2021:5099
Version-Release number of selected component (if applicable):
Rhel 8.4 EUS
systemd-239-45.el8_4.6.x86_64
How reproducible:
(Note we've only seen this once so far, so there is no way to easily reproduce this)
Additional info:
[root@sleola sosreport-srv009201-2022-01-25-xjnnjeu]# cat installed-rpms | grep -i systemd
python3-systemd-234-8.el8.x86_64 Thu May 13 14:53:43 2021
rpm-plugin-systemd-inhibit-4.14.3-14.el8_4.x86_64 Thu Jul 15 19:42:01 2021
systemd-239-45.el8_4.6.x86_64 Fri Jan 14 19:23:40 2022 <-- updated at Jan 14Th
systemd-libs-239-45.el8_4.6.x86_64 Fri Jan 14 19:23:39 2022
systemd-pam-239-45.el8_4.6.x86_64 Fri Jan 14 19:23:40 2022
systemd-udev-239-45.el8_4.6.x86_64 Fri Jan 14 19:23:41 2022
[root@sleola sosreport-srv009201-2022-01-25-xjnnjeu]#
[root@sleola sosreport-srv009201-2022-01-25-xjnnjeu]# sed -n 37,45p sos_commands/abrt/abrt-cli_list
id dbd03a18e86b17ad8f9eaccc99ad2fc9aa773d83
reason: realloc(): systemd killed by SIGABRT
time: Tue Jan 25 16:21:01 2022 <--- Crash happened after using the version that should have the fixes
cmdline: /usr/lib/systemd/systemd --switched-root --system --deserialize 18
package: systemd-239-45.el8_4.6
uid: 0 (root)
count: 1
Directory: /var/spool/abrt/ccpp-2022-01-25-16:21:01-2137021
Run 'abrt-cli report /var/spool/abrt/ccpp-2022-01-25-16:21:01-2137021' for creating a case in Red Hat Customer Portal
[root@sleola sosreport-srv009201-2022-01-25-xjnnjeu]#
(gdb) bt
#0 0x00007ff8860c067b in kill () at ../sysdeps/unix/syscall-template.S:78
#1 0x000055b56f246f7a in crash (sig=6) at ../src/core/main.c:194
#2 <signal handler called>
#3 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#4 0x00007ff8860aadb5 in __GI_abort () at abort.c:79
#5 0x00007ff8861034e7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ff886212a0e "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#6 0x00007ff88610a5ec in malloc_printerr (str=str@entry=0x7ff886214a88 "malloc(): smallbin double linked list corrupted") at malloc.c:5374
#7 0x00007ff88610d8b4 in _int_malloc (av=av@entry=0x7ff886448bc0 <main_arena>, bytes=bytes@entry=113) at malloc.c:3656
#8 0x00007ff88610e3af in _int_realloc (av=av@entry=0x7ff886448bc0 <main_arena>, oldp=oldp@entry=0x55b56fbb54b0, oldsize=oldsize@entry=112, nb=nb@entry=128) at malloc.c:4612
#9 0x00007ff88610f67b in __GI___libc_realloc (oldmem=0x55b56fbb54c0, bytes=bytes@entry=112) at malloc.c:3238
#10 0x00007ff88789d1f6 in message_extend_fields (m=m@entry=0x55b56fbfff40, align=align@entry=8, sz=sz@entry=7, add_offset=add_offset@entry=false) at ../src/libsystemd/sd-bus/bus-message.c:163
#11 0x00007ff8878a57fa in message_append_field_signature (h=8, ret=0x0, s=0x55b56fc124a0 "s", m=0x55b56fbfff40) at ../src/libsystemd/sd-bus/bus-message.c:299
#12 sd_bus_message_seal (m=m@entry=0x55b56fbfff40, cookie=4070336, timeout_usec=timeout_usec@entry=25000000) at ../src/libsystemd/sd-bus/bus-message.c:2922
#13 0x00007ff8878b767e in bus_seal_message (b=b@entry=0x55b56fbcffd0, m=0x55b56fbfff40, timeout=25000000, timeout@entry=0) at ../src/libsystemd/sd-bus/sd-bus.c:1667
#14 0x00007ff8878be7e9 in sd_bus_send (bus=0x55b56fbcffd0, _m=<optimized out>, cookie=cookie@entry=0x0) at ../src/libsystemd/sd-bus/sd-bus.c:1860
#15 0x00007ff887890179 in sd_bus_reply_method_error (call=0x55b56fc67840, e=<optimized out>) at ../src/libsystemd/sd-bus/bus-convenience.c:185
#16 0x00007ff887890673 in sd_bus_reply_method_errno (call=0x55b56fc67840, error=-3, p=0x7ffc4df230d0) at ../src/libsystemd/sd-bus/bus-convenience.c:236
#17 0x00007ff88789a09d in bus_maybe_reply_error (m=m@entry=0x55b56fc67840, r=r@entry=-3, error=error@entry=0x7ffc4df230d0) at ../src/libsystemd/sd-bus/bus-internal.c:341
#18 0x00007ff8878ad4c0 in method_callbacks_run (found_object=0x7ffc4df23187, require_fallback=false, c=<optimized out>, m=0x55b56fc67840, bus=0x55b56fbcffd0) at ../src/libsystemd/sd-bus/bus-objects.c:409
#19 object_find_and_run (bus=0x55b56fbcffd0, m=0x55b56fc67840, p=0x55b56fc61198 "/org/freedesktop/systemd1", require_fallback=false, found_object=0x7ffc4df23187) at ../src/libsystemd/sd-bus/bus-objects.c:1268
#20 0x00007ff8878aec7c in bus_process_object (bus=bus@entry=0x55b56fbcffd0, m=m@entry=0x55b56fc67840) at ../src/libsystemd/sd-bus/bus-objects.c:1388
#21 0x00007ff8878bdbaa in process_message (m=0x55b56fc67840, bus=0x55b56fbcffd0) at ../src/libsystemd/sd-bus/sd-bus.c:2705
#22 process_running (ret=0x0, priority=<optimized out>, hint_priority=<optimized out>, bus=0x55b56fbcffd0) at ../src/libsystemd/sd-bus/sd-bus.c:2747
#23 bus_process_internal (bus=0x55b56fbcffd0, hint_priority=<optimized out>, priority=<optimized out>, ret=0x0) at ../src/libsystemd/sd-bus/sd-bus.c:2966
#24 0x00007ff8878bf89c in io_callback (s=<optimized out>, fd=<optimized out>, revents=<optimized out>, userdata=0x55b56fbcffd0) at ../src/libsystemd/sd-bus/sd-bus.c:3346
#25 0x00007ff8878e6f19 in source_dispatch (s=0x55b56fc1b790) at ../src/libsystemd/sd-event/sd-event.c:3344
#26 0x00007ff8878e8d0d in sd_event_dispatch (e=0x55b56f9fb520) at ../src/libsystemd/sd-event/sd-event.c:3763
#27 0x00007ff8878e8f18 in sd_event_run (e=0x55b56f9fb520, timeout=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:3823
#28 0x000055b56f28bab2 in manager_loop (m=m@entry=0x55b56f9fd560) at ../src/core/manager.c:2884
#29 0x000055b56f242471 in invoke_main_loop (ret_error_message=0x7ffc4df23548, ret_switch_root_init=<synthetic pointer>, ret_switch_root_dir=<synthetic pointer>, ret_fds=0x7ffc4df23558, ret_shutdown_verb=<synthetic pointer>,
ret_retval=<synthetic pointer>, ret_reexecute=<synthetic pointer>, saved_rlimit_memlock=0x7ffc4df235c0, saved_rlimit_nofile=0x7ffc4df235b0, m=0x55b56f9fd560) at ../src/core/main.c:1665
#30 main (argc=<optimized out>, argv=0x7ffc4df23828) at ../src/core/main.c:2606
(gdb)
If the customer wants to help us, here is a new test build:
https://people.redhat.com/~lnykryn/systemd-239-46.el8_4.9.1.asan/
It is built with asan, based on the latest 8.4 z-stream and I've added the patch Frantisek found. What we got in the log is either a bug and then the build could fix it or a false positive and then it should remove it.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (systemd bug fix and enhancement update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2022:7727