Cause: gnutls detection of AVX instructions does not function correctly if x86 extended register state save and restore is disabled, e.g., with a `noxsave` kernel parameter
Consequence: gnutls crashes with a `trap invalid opcode` error message
Fix: gnutls abstains from using AVX-accelerated code if XSAVE is not detected
Result: disabling XSAVE on AVX-enabled hosts doesn't cause gnutls to crash
Description of problem: When customer enables FIPS on RHEL 8.6 (VMWare ESXI Server 7.0.0, CPU- intel i9-10980XE) it leads to multiple SIGILL service failures related to libgnutls.so.30.28.2. Multiple services are failing UNIT LOAD ACTIVE SUB DESCRIPTION * cups.path loaded failed failed CUPS Scheduler * accounts-daemon.service loaded failed failed Accounts Service * chronyd.service loaded failed failed NTP client/server * cups.service loaded failed failed CUPS Scheduler * firewalld.service loaded failed failed firewalld - dynamic firewall daemon * fprintd.service loaded failed failed Fingerprint Authentication Daemon * gdm.service loaded failed failed GNOME Display Manager All services are failing with similar Illegal Instruction related to gnutls library. Jul 26 09:43:23 fixme kernel: traps: pool[6677] trap invalid opcode ip:7feef81809fe sp:7fee997419c0 error:0 in libgnutls.so.30.28.2[7feef8040000+1dd000] Jul 26 09:47:34 fixme kernel: traps: fwupd[4666] trap invalid opcode ip:7ff45d02e9bb sp:7ffd2dcd6340 error:0 in libgnutls.so.30.28.2[7ff45cef3000+1dd000] Jul 26 09:49:16 fixme kernel: traps: NetworkManager[7750] trap invalid opcode ip:7ff828e7b9bb sp:7ffd5c110d00 error:0 in libgnutls.so.30.28.2[7ff828d40000+1dd000] Jul 26 09:50:11 fixme kernel: traps: nm-initrd-gener[407] trap invalid opcode ip:7fd8f313c9bb sp:7ffcd0833960 error:0 in libgnutls.so.30.28.2[7fd8f3001000+1dd000] Jul 26 09:50:17 fixme kernel: traps: ostree-system-g[1044] trap invalid opcode ip:7f532c31e9bb sp:7fff82ee52f0 error:0 in libgnutls.so.30.28.2[7f532c1e3000+1dd000] Jul 26 09:50:17 fixme kernel: traps: libinput-device[1117] trap invalid opcode ip:7f5b7649e9bb sp:7fff5f22d870 error:0 in libgnutls.so.30.28.2[7f5b76363000+1dd000] Jul 26 09:50:17 fixme kernel: traps: rename_device[1122] trap invalid opcode ip:7f109da189bb sp:7ffeec7356d0 error:0 in libgnutls.so.30.28.2[7f109d8dd000+1dd000] Jul 26 09:52:18 fixme kernel: traps: irqbalance[1264] trap invalid opcode ip:7f3bb29419bb sp:7fff61bf5cf0 error:0 in libgnutls.so.30.28.2[7f3bb2806000+1dd000] var/log/messages:Jul 26 09:52:18 fixme kernel: traps: VGAuthService[1250] trap invalid opcode ip:7f17b56349bb sp:7ffdf1c91460 error:0 in libgnutls.so.30.28.2[7f17b54f9000+1dd000] Jul 26 09:52:18 fixme kernel: traps: chronyd[1249] trap invalid opcode ip:7fbeed47a9bb sp:7ffe3156df10 error:0 in libgnutls.so.30.28.2[7fbeed33f000+1dd000] Jul 26 09:52:19 fixme kernel: traps: vmtoolsd[1251] trap invalid opcode ip:7f776159d9bb sp:7ffe0dfd5740 error:0 in libgnutls.so.30.28.2[7f7761462000+1dd000] Jul 26 09:52:19 fixme kernel: traps: polkitd[1259] trap invalid opcode ip:7f212eb7f9bb sp:7ffdbe375df0 error:0 in libgnutls.so.30.28.2[7f212ea44000+1dd000] Application cores are dumped for every failure of the service. $ for file in ./*; do echo "==== $file ====="; eu-unstrip -n --core=./$file | head -1; done ==== ./core.1120 ===== 0x55b383619000+0x204000 1cefa16ca358662c49252fb994385ea710882c7b@0x55b383619318 . - /usr/lib/udev/libinput-device-group ==== ./core.1123 ===== 0x55eabe18b000+0x204000 16afc62b49176225a44c21534425ae24a0489f9e@0x55eabe18b318 . - /usr/lib/udev/rename_device ==== ./core.1125 ===== 0x558a1ca5c000+0x204000 1cefa16ca358662c49252fb994385ea710882c7b@0x558a1ca5c318 . - /usr/lib/udev/libinput-device-group ==== ./core.1244 ===== 0x555ab63d9000+0x222000 7ba8bbe851de95c165699696bfeb126e7b8e9511@0x555ab63d9318 . - /usr/bin/VGAuthService ==== ./core.1245 ===== 0x55ba766ea000+0x212000 85bce56080f9749a780aa591c7a3d258d33d3184@0x55ba766ea318 . - /usr/bin/vmtoolsd ==== ./core.1250 ===== 0x55561a453000+0x279000 758c98101436dfbee5e6ce4db5d68f74736ada4f@0x55561a453350 . - /usr/libexec/udisks2/udisksd ==== ./core.1269 ===== 0x55836fc92000+0x203000 81906b98278d4eed5faafd38499331b4c56daa50@0x55836fc92284 . - /usr/libexec/platform-python3.6 ==== ./core.1285 ===== 0x55caa8314000+0x569000 e128c83e67fd41b39f0938e628d4a669bd134284@0x55caa8314350 . - /usr/sbin/NetworkManager ==== ./core.1295 ===== 0x555822248000+0x206000 8b767f9bbf3814840747c2d8963ac3b745d1f188@0x555822248318 . - /usr/bin/rhsmcertd ==== ./core.1300 ===== 0x5615bd7ee000+0x271000 ddfff834b1549302a0597251b8547cfdd8421bd6@0x5615bd7ee318 . - /usr/sbin/cupsd ==== ./core.1389 ===== 0x558994f4e000+0x271000 ddfff834b1549302a0597251b8547cfdd8421bd6@0x558994f4e318 . - /usr/sbin/cupsd ==== ./core.1396 ===== 0x55e46d6b8000+0x27c000 aaa3aec7fde3dab9bb6de50c799ab300d24ffee9@0x55e46d6b8318 . - /usr/sbin/libvirtd ==== ./core.1398 ===== 0x563a68762000+0x270000 5b56d0af4627ef298324dd406d14966c60017f51@0x563a68762318 . - /usr/sbin/gdm ==== ./core.1491 ===== 0x55c44da48000+0x270000 5b56d0af4627ef298324dd406d14966c60017f51@0x55c44da48318 . - /usr/sbin/gdm 0x55666f696000+0x270000 5b56d0af4627ef298324dd406d14966c60017f51@0x55666f696318 . - /usr/sbin/gdm ==== ./core.1513 ===== 0x564e78466000+0x27c000 aaa3aec7fde3dab9bb6de50c799ab300d24ffee9@0x564e78466318 . - /usr/sbin/libvirtd ==== ./core.1514 ===== 0x558ce2347000+0x270000 5b56d0af4627ef298324dd406d14966c60017f51@0x558ce2347318 . - /usr/sbin/gdm ==== ./core.1522 ===== 0x561de0adf000+0x270000 5b56d0af4627ef298324dd406d14966c60017f51@0x561de0adf318 . - /usr/sbin/gdm ==== ./core.1523 ===== 0x55f0fe451000+0x271000 ddfff834b1549302a0597251b8547cfdd8421bd6@0x55f0fe451318 . - /usr/sbin/cupsd ==== ./core.1531 ===== 0x55f14440c000+0x27c000 aaa3aec7fde3dab9bb6de50c799ab300d24ffee9@0x55f14440c318 . - /usr/sbin/libvirtd ==== ./core.1539 ===== 0x55ad61ec1000+0x27c000 aaa3aec7fde3dab9bb6de50c799ab300d24ffee9@0x55ad61ec1318 . - /usr/sbin/libvirtd ==== ./core.1540 ===== 0x5593f0aaf000+0x271000 ddfff834b1549302a0597251b8547cfdd8421bd6@0x5593f0aaf318 . - /usr/sbin/cupsd ==== ./core.1747 ===== 0x560cb7eb5000+0x391000 32087fea0eb14d63551785f4fffd2c573c82605b@0x560cb7eb5318 . - /usr/bin/flatpak > The services are dumping core and all these are related to gnutls library with same error logs. traps: service[PID] trap invalid opcode ip:7f17b56349bb sp:7ffdf1c91460 error:0 in libgnutls.so.30.28.2 Version-Release number of selected component (if applicable): gnutls-3.6.16-4.el8.x86_64 How reproducible: After upgrading from RHEL 8.5 to 8.6, first boot would show multiple failing services (process core dumps) and invalid opcode messages (libgnutls.so.30.28.2). Steps to Reproduce: 1. Upgrade RHEL 8.5 to 8.6 or 2. Install fresh RHEL 8.6 3. Enable FIPS mode 4. Reboot the system Actual results: - Multiple services fails to start with core dump. traps: service[PID] trap invalid opcode ip:7f17b56349bb sp:7ffdf1c91460 error:0 in libgnutls.so.30.28.2 Expected results: - There should be no such service failures in FIPS mode. Additional info: