Bug 2232099

Summary:	AMD ucode updates not applied on boot
Product:	[Fedora] Fedora	Reporter:	Ralf Ertzinger <redhat-bugzilla>
Component:	kernel	Assignee:	Kernel Maintainer List <kernel-maint>
Status:	NEW ---	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	38	CC:	acaringi, adscvr, airlied, alciregi, bskeggs, dwmw2, hdegoede, hpa, jarod, jforbes, josef, jwboyer, kernel-maint, laura, lgoncalv, linville, masami256, mchehab, pbrobinson, ptalbert, steved
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ralf Ertzinger 2023-08-15 09:48:33 UTC

Note: I'm not certain about the correct component for this issue, I'm starting here as this is where Fedora ships AMD CPU ucode files.

I have an AMD CPU system where ucode updates do not get applied on boot.

I've verified that

* There is a ucode update for the CPU available
* It is shipped in /lib/firmware/amd-ucode/microcode_amd_fam17h.bin
* It is present in the early initrd in kernel/x86/microcode/AuthenticAMD.bin

Nonetheless, there is no indication in dmesg after a reboot that a ucode update was applied, and /proc/cpuinfo still shows an outdated ucode version.

I've never had any trouble with this on Intel CPUs, and I'm not clear on what, if anything, I'm doing wrong.

Reproducible: Always

Comment 1 Peter Robinson 2023-08-15 10:04:05 UTC

The kernel loads the microcode (arch/x86/kernel/cpu/microcode) so as long as it's shipped in the distro it's nothing to do with linux-firmware, it may be an issue with dracut as it deals with getting into the initrd for the kernel to deal with.

Probably useful to have versions of kernel/linux-firmware/dracut on the bug for reference.

Comment 2 Ralf Ertzinger 2023-08-15 10:21:07 UTC

Current versions:

kernel-6.4.8-200.fc38.x86_64
dracut-059-3.fc38.x86_64
linux-firmware-20230625-151.fc38.noarch

The CPU ID is 00800f12 (as shown by cpuid 1/eax), the microcode shipped in the BIOS is version 0x08001227:

```
[    6.161195] microcode: CPU0: patch_level=0x08001227
[    6.161197] microcode: CPU1: patch_level=0x08001227
[    6.161198] microcode: CPU4: patch_level=0x08001227
[    6.161199] microcode: CPU5: patch_level=0x08001227
[    6.161199] microcode: CPU2: patch_level=0x08001227
[    6.161199] microcode: CPU3: patch_level=0x08001227
[    6.161201] microcode: CPU6: patch_level=0x08001227
[    6.161204] microcode: CPU7: patch_level=0x08001227
```

Extracting the microcode file from the initrd and inspecting it:


```
[sun ethan ~ :) 26]$ cpuid -1 -l 1 -r
CPU:
   0x00000001 0x00: eax=0x00800f12 ebx=0x08080800 ecx=0x7ed8320b edx=0x178bfbff
[sun ethan ~ :) 27]$ mkdir /tmp/foo
[sun ethan ~ :) 28]$ cd /tmp/foo/
[sun ethan /tmp/foo :) 30]$ sudo cat /boot/initramfs-6.4.8-200.fc38.x86_64.img | cpio -id
102 blocks
[sun ethan /tmp/foo :) 31]$ ~/venv/MCE/bin/python ~/src/GIT/MCExtractor/MCE.py kernel/x86/microcode/AuthenticAMD.bin -skip -exit
╔═══════════════════════════════════════════╗
║         MC Extractor v1.90.2 r274         ║
╚═══════════════════════════════════════════╝

AuthenticAMD.bin (1/1)

╔════════════════════════════════════════════════════════════════╗
║                              AMD                               ║
╟────┬──────────┬──────────┬────────────┬────────┬────────┬──────╢
║ #  │  CPUID   │ Revision │    Date    │  Size  │ Offset │ Last ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 1  │ 00100F22 │ 01000083 │ 2008-04-30 │ 0x3C0  │ 0x380  │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 2  │ 00100F20 │ 01000084 │ 2008-05-01 │ 0x3C0  │ 0x748  │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 3  │ 00100F62 │ 010000C7 │ 2010-03-11 │ 0x3C0  │ 0xB10  │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 4  │ 00100F43 │ 010000C8 │ 2010-03-11 │ 0x3C0  │ 0xED8  │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 5  │ 00100F81 │ 010000D9 │ 2011-10-12 │ 0x3C0  │ 0x12A0 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 6  │ 00100F80 │ 010000DA │ 2011-10-24 │ 0x3C0  │ 0x1668 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 7  │ 00100F41 │ 010000DB │ 2011-10-24 │ 0x3C0  │ 0x1A30 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 8  │ 00100FA0 │ 010000DC │ 2011-10-24 │ 0x3C0  │ 0x1DF8 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 9  │ 00200F31 │ 02000032 │ 2008-03-06 │ 0x200  │ 0x21C0 │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 10 │ 00300F10 │ 03000027 │ 2011-13-09 │ 0x3C0  │ 0x23C8 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 11 │ 00500F10 │ 05000029 │ 2013-01-21 │ 0x620  │ 0x2790 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 12 │ 00500F20 │ 05000119 │ 2013-01-18 │ 0x620  │ 0x2DB8 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 13 │ 00600F12 │ 0600063E │ 2018-02-07 │ 0xA20  │ 0x342C │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 14 │ 00600F20 │ 06000852 │ 2018-02-06 │ 0xA20  │ 0x3E54 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 15 │ 00610F01 │ 06001119 │ 2012-07-13 │ 0xA20  │ 0x487C │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 16 │ 00700F01 │ 0700010F │ 2014-02-19 │ 0xD60  │ 0x52D0 │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 17 │ 00800F82 │ 0800820D │ 2019-04-16 │ 0xC80  │ 0x60A6 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 18 │ 00800F12 │ 0800126E │ 2021-11-11 │ 0xC80  │ 0x6D2E │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 19 │ 00830F10 │ 08301072 │ 2022-02-15 │ 0xC80  │ 0x79B6 │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 20 │ 00A00F10 │ 0A001078 │ 2023-01-17 │ 0x15C0 │ 0x868A │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 21 │ 00A00F11 │ 0A0011CE │ 2023-01-14 │ 0x15C0 │ 0x9C52 │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼────────┼──────╢
║ 22 │ 00A00F12 │ 0A001231 │ 2023-01-17 │ 0x15C0 │ 0xB21A │  No  ║
╚════╧══════════╧══════════╧════════════╧════════╧════════╧══════╝
```

The tool is MCExtractor from https://github.com/platomav/MCExtractor
Line 18 shows there is an update for this CPU available in the file with a version of 0x0800126E

Comment 3 Peter Robinson 2023-08-15 10:48:51 UTC

> linux-firmware-20230625-151.fc38.noarch

There's newer AMD micro code in both -152 and -153 linux-firmware update.

It would be worthwhile updating to that and checking the firmware in /lib/firmware/amd-ucode/ and see if it differs from what you're expecting.

Comment 4 Ralf Ertzinger 2023-08-15 11:24:19 UTC

Updated system:

kernel-6.4.10-200.fc38.x86_64
dracut-059-3.fc38.x86_64
linux-firmware-20230804-153.fc38.noarch

```
[sun ethan ~ :) 10]$ cat /proc/cpuinfo | grep microcode
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
microcode       : 0x8001227
[sun ethan ~ :) 11]$ uname -r
6.4.10-200.fc38.x86_64
[sun ethan ~ :) 12]$ mkdir /tmp/foo
[sun ethan ~ :) 13]$ cd /tmp/foo/
[sun ethan /tmp/foo :) 14]$ sudo cat /boot/initramfs-6.4.10-200.fc38.x86_64.img | cpio -id
153 blocks
[sun ethan /tmp/foo :) 15]$ ~/venv/MCE/bin/python ~/src/GIT/MCExtractor/MCE.py kernel/x86/microcode/AuthenticAMD.bin -skip -exit
╔═══════════════════════════════════════════╗
║         MC Extractor v1.90.2 r274         ║
╚═══════════════════════════════════════════╝

AuthenticAMD.bin (1/1)

╔═════════════════════════════════════════════════════════════════╗
║                               AMD                               ║
╟────┬──────────┬──────────┬────────────┬────────┬─────────┬──────╢
║ #  │  CPUID   │ Revision │    Date    │  Size  │  Offset │ Last ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 1  │ 00100F22 │ 01000083 │ 2008-04-30 │ 0x3C0  │  0x4BC  │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 2  │ 00100F20 │ 01000084 │ 2008-05-01 │ 0x3C0  │  0x884  │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 3  │ 00100F62 │ 010000C7 │ 2010-03-11 │ 0x3C0  │  0xC4C  │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 4  │ 00100F43 │ 010000C8 │ 2010-03-11 │ 0x3C0  │  0x1014 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 5  │ 00100F81 │ 010000D9 │ 2011-10-12 │ 0x3C0  │  0x13DC │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 6  │ 00100F80 │ 010000DA │ 2011-10-24 │ 0x3C0  │  0x17A4 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 7  │ 00100F41 │ 010000DB │ 2011-10-24 │ 0x3C0  │  0x1B6C │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 8  │ 00100FA0 │ 010000DC │ 2011-10-24 │ 0x3C0  │  0x1F34 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 9  │ 00200F31 │ 02000032 │ 2008-03-06 │ 0x200  │  0x22FC │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 10 │ 00300F10 │ 03000027 │ 2011-13-09 │ 0x3C0  │  0x2504 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 11 │ 00500F10 │ 05000029 │ 2013-01-21 │ 0x620  │  0x28CC │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 12 │ 00500F20 │ 05000119 │ 2013-01-18 │ 0x620  │  0x2EF4 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 13 │ 00600F12 │ 0600063E │ 2018-02-07 │ 0xA20  │  0x3568 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 14 │ 00600F20 │ 06000852 │ 2018-02-06 │ 0xA20  │  0x3F90 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 15 │ 00610F01 │ 06001119 │ 2012-07-13 │ 0xA20  │  0x49B8 │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 16 │ 00700F01 │ 0700010F │ 2014-02-19 │ 0xD60  │  0x540C │  No  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 17 │ 00800F82 │ 0800820D │ 2019-04-16 │ 0xC80  │  0x61F2 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 18 │ 00830F10 │ 0830107A │ 2023-05-17 │ 0xC80  │  0x6E7A │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 19 │ 008A0F00 │ 08A00008 │ 2023-06-15 │ 0xC80  │  0x7B02 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 20 │ 00800F12 │ 0800126E │ 2021-11-11 │ 0xC80  │  0x878A │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 21 │ 00A10F11 │ 0A10113E │ 2023-06-20 │ 0x15C0 │  0x949E │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 22 │ 00A10F12 │ 0A10123E │ 2023-06-20 │ 0x15C0 │  0xAA66 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 23 │ 00AA0F02 │ 0AA00212 │ 2023-06-19 │ 0x15C0 │  0xC02E │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 24 │ 00A00F11 │ 0A0011D1 │ 2023-07-10 │ 0x15C0 │  0xD5F6 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 25 │ 00A00F10 │ 0A001079 │ 2023-06-09 │ 0x15C0 │  0xEBBE │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 26 │ 00A00F12 │ 0A001234 │ 2023-07-10 │ 0x15C0 │ 0x10186 │ Yes  ║
╟────┼──────────┼──────────┼────────────┼────────┼─────────┼──────╢
║ 27 │ 00AA0F01 │ 0AA00116 │ 2023-06-19 │ 0x15C0 │ 0x1174E │ Yes  ║
╚════╧══════════╧══════════╧════════════╧════════╧═════════╧══════╝
```
While there were updates in the microcode file, none of them applied to this CPU, the latest version in the file is still 0x0800126E (line 20)