Description of problem: Yum crashes after glibc update to version 2.28-145. [root@localhost ~]# yum search python Illegal instruction (core dumped) [root@localhost ~]# dmesg | tail -5 [ 726.087955] traps: yum[1696] trap invalid opcode ip:7f390bbfa59f sp:7ffee22d19a0 error:0 in libm-2.28.so[7f390bb72000+181000] [ 773.118330] traps: yum[1730] trap invalid opcode ip:7f86d6a8c59f sp:7ffc07700320 error:0 in libm-2.28.so[7f86d6a04000+181000] [ 839.974365] traps: yum[1744] trap invalid opcode ip:7f71ee5fc59f sp:7ffd948b36a0 error:0 in libm-2.28.so[7f71ee574000+181000] [ 870.615860] traps: yum[1755] trap invalid opcode ip:7fe4ea5d559f sp:7ffe5ac21760 error:0 in libm-2.28.so[7fe4ea54d000+181000] [ 899.107078] traps: yum[1767] trap invalid opcode ip:7f0465e5859f sp:7ffdfbacafe0 error:0 in libm-2.28.so[7f0465dd0000+181000] [root@localhost ~]# Version-Release number of selected component (if applicable): glibc-2.28-145.el8.x86_64 glibc-2.28-145.el8.i686 How reproducible: Problem seen on some processors. Following is the processor info on the problem machine. /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 142 model name : Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz stepping : 10 microcode : 0xb4 cpu MHz : 2112.002 cache size : 8192 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 22 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon nopl tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx hypervisor lahf_lm 3dnowprefetch pti arat bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit bogomips : 4224.00 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: Steps to Reproduce: 1. Update to glibc 2.28-145 2. Run any yum command 3. Actual results: Expected results: Additional info:
The CPU flags do not match the CPU model, so I assume that virtualization is involved. What is your hypervisor, and how have you configured it? Would you please check if systemd-coredumpd has obtained a coredump, and try to get a backtrace and disassembly using “coredumpctl gdb”? Thanks.
1. Yes. This is virtual machine running on a windows host. I am using the vmware workstation (15.X). Are there any specific settings that you are interested in. 2. Following is the trace obtained with "coredumpctl gdb" [root@localhost sysctl.d]# coredumpctl gdb PID: 10417 (yum) UID: 0 (root) GID: 0 (root) Signal: 4 (ILL) Timestamp: Wed 2021-01-20 19:09:55 IST (1min 45s ago) Command Line: /usr/libexec/platform-python /usr/bin/yum search python Executable: /usr/libexec/platform-python3.6 Control Group: /user.slice/user-0.slice/session-1.scope Unit: session-1.scope Slice: user-0.slice Session: 1 Owner UID: 0 (root) Boot ID: 4c95db69a0a34fe591a4ffb589bac207 Machine ID: cb9c7789d294492691e347bdbd1c181e Hostname: localhost.localdomain Storage: /var/lib/systemd/coredump/core.yum.0.4c95db69a0a34fe591a4ffb589bac207.10417.1611149995000000.lz4 Message: Process 10417 (yum) of user 0 dumped core. Stack trace of thread 10417: #0 0x00007fe6c05b359f __exp1_fma4 (libm.so.6) #1 0x00007fe6c05b4384 __ieee754_pow_fma4.localalias.0 (libm.so.6) #2 0x00007fe6c053a408 powf32x (libm.so.6) #3 0x00007fe6c104e3f8 float_pow (libpython3.6m.so.1.0) #4 0x00007fe6c102fb3c long_pow (libpython3.6m.so.1.0) #5 0x00007fe6c100b2f3 ternary_op.isra.39 (libpython3.6m.so.1.0) #6 0x00007fe6c10749a7 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #7 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #8 0x00007fe6c0fd02b3 PyEval_EvalCode (libpython3.6m.so.1.0) #9 0x00007fe6c10e0f20 builtin_exec (libpython3.6m.so.1.0) #10 0x00007fe6c106b502 PyCFunction_Call (libpython3.6m.so.1.0) #11 0x00007fe6c1078bfe _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #12 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #13 0x00007fe6c104f4f0 fast_function (libpython3.6m.so.1.0) #14 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #15 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #16 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #17 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #18 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #19 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #20 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #21 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #22 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #23 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #24 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #25 0x00007fe6c0fd0422 _PyFunction_FastCallDict (libpython3.6m.so.1.0) #26 0x00007fe6c0fd11fe _PyObject_FastCallDict (libpython3.6m.so.1.0) #27 0x00007fe6c10b5a7e _PyObject_CallMethodIdObjArgs (libpython3.6m.so.1.0) #28 0x00007fe6c0fd18a4 PyImport_ImportModuleLevelObject (libpython3.6m.so.1.0) #29 0x00007fe6c1077219 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #30 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #31 0x00007fe6c0fd02b3 PyEval_EvalCode (libpython3.6m.so.1.0) #32 0x00007fe6c10e0f20 builtin_exec (libpython3.6m.so.1.0) #33 0x00007fe6c106b502 PyCFunction_Call (libpython3.6m.so.1.0) #34 0x00007fe6c1078bfe _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #35 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #36 0x00007fe6c104f4f0 fast_function (libpython3.6m.so.1.0) #37 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #38 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #39 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #40 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #41 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #42 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #43 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #44 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #45 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #46 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #47 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #48 0x00007fe6c0fd0422 _PyFunction_FastCallDict (libpython3.6m.so.1.0) #49 0x00007fe6c0fd11fe _PyObject_FastCallDict (libpython3.6m.so.1.0) #50 0x00007fe6c10b5a7e _PyObject_CallMethodIdObjArgs (libpython3.6m.so.1.0) #51 0x00007fe6c0fd18a4 PyImport_ImportModuleLevelObject (libpython3.6m.so.1.0) #52 0x00007fe6c1077219 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #53 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #54 0x00007fe6c0fd02b3 PyEval_EvalCode (libpython3.6m.so.1.0) #55 0x00007fe6c10e0f20 builtin_exec (libpython3.6m.so.1.0) #56 0x00007fe6c106b502 PyCFunction_Call (libpython3.6m.so.1.0) #57 0x00007fe6c1078bfe _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #58 0x00007fe6c0fcef14 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0) #59 0x00007fe6c104f4f0 fast_function (libpython3.6m.so.1.0) #60 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) #61 0x00007fe6c1073488 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0) #62 0x00007fe6c104f308 fast_function (libpython3.6m.so.1.0) #63 0x00007fe6c1072847 call_function (libpython3.6m.so.1.0) GNU gdb (GDB) Red Hat Enterprise Linux 8.2-15.el8 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/platform-python3.6...Reading symbols from .gnu_debugdata for /usr/libexec/platform-python3.6...(no debugging symbols found)...done. (no debugging symbols found)...done. warning: core file may not match specified executable file. [New LWP 10417] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/usr/libexec/platform-python /usr/bin/yum search python'. Program terminated with signal SIGILL, Illegal instruction. #0 0x00007fe6c05b359f in __exp1_fma4 () from /lib64/libm.so.6 Missing separate debuginfos, use: yum debuginfo-install platform-python-3.6.8-33.el8.x86_64 (gdb)
Thanks, the backtrace was illuminating. We need to backport this upstream commit: commit 23af890b3f04e80da783ba64e6b6d94822e01d54 Author: Ondřej Hošek <ondra.hosek> Date: Wed Aug 26 04:26:50 2020 +0200 x86-64: Fix FMA4 detection in ifunc [BZ #26534] A typo in commit 107e6a3c2212ba7a3a4ec7cae8d82d73f7c95d0b causes the FMA4 code path to be taken on systems that support FMA, even if they do not support FMA4. Fix this to detect FMA4.
Hi Is there a tentative date for the new package to get published on Centos Stream repository? Thanks #psm
(In reply to P. Shesh Murthy from comment #6) > Is there a tentative date for the new package to get published on Centos > Stream repository? I expect an update of CentOS Stream in mid-February, as the new build has to pass through QE first. I have uploaded repository with a hotfix build: http://people.redhat.com/~fweimer/HorTWjurJ7Qk/glibc-2.28-146.el8.0.bz1918115.0/ As a workaround, you can select a different machine model in the hypervisor. Anything that advertises AVX2 to the guest should work. As a sight effect, you should also get better performance from your VMs. A machine model that does not indicate support for FMA will work, too. The trigger for this bug is that the CPUID information in the guest has the FMA bit set, but the virtualized CPU lacks support for XSAVE, so that the kernel cannot enable it. This case should still be handled correctly in the sense that there are no crashes, of course, but it also should not happen in practice (neither on bare metal, nor with properly configured hypervisor machine models).
Thanks for the information. My vmware machine was running in ESX 5.0 compat mode. Once I updated the machine to ESX 6.0, I see the avx2 flag in /proc/cpuinfo and the problem is no longer observed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: glibc security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:1585