Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1753541

Summary: [rhel-7.8] Update Intel microcode version to microcode-20190918
Product: Red Hat Enterprise Linux 7 Reporter: Eugene Syromiatnikov <esyr>
Component: microcode_ctlAssignee: Eugene Syromiatnikov <esyr>
Status: CLOSED ERRATA QA Contact: Jeff Bastian <jbastian>
Severity: medium Docs Contact:
Priority: high    
Version: 7.7CC: cshao, mthacker, mvanderw, skozina
Target Milestone: rcKeywords: ZStream
Target Release: 7.8   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: microcode_ctl-2.1-54.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1758567 1758568 1758569 1758570 1758571 1758572 (view as bug list) Environment:
Last Closed: 2020-03-31 20:08:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1710953, 1758567, 1758568, 1758569, 1758570, 1758571, 1758572, 1784906    

Description Eugene Syromiatnikov 2019-09-19 09:14:22 UTC
There is a new Intel microcode release[1], that is to be packaged.

[1] https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/releases/tag/microcode-20190918

Comment 1 Eugene Syromiatnikov 2019-09-19 09:18:15 UTC
microcode-20190918 release includes the following microcode updates:

Processor             Identifier     Version       Products
Model        Stepping F-MO-S/PI      Old->New
BDW-U/Y      E0/F0    6-3d-4/c0 0000002d->0000002e Core Gen5
HSX-EX       E0       6-3f-4/80 00000014->00000016 Xeon E7 v3
BDW-H/E3     E0/G0    6-47-1/22 00000020->00000021 Core Gen5
BDX-ML       B0/M0/R0 6-4f-1/ef 0b000036->0b000038 Xeon E5/E7 v4; Core i7-69xx/68xx
BDX-DE       V1       6-56-2/10 0000001a->0000001c Xeon D-1520/40
BDX-DE       V2/3     6-56-3/10 07000017->07000019 Xeon D-1518/19/21/27/28/31/33/37/41/48, Pentium D1507/08/09/17/19
BDX-DE       Y0       6-56-4/10 0f000015->0f000017 Xeon D-1557/59/67/71/77/81/87
BDX-NS       A0       6-56-5/10 0e00000d->0e00000f Xeon D-1513N/23/33/43/53
SKX-SP       H0/M0/U0 6-55-4/b7 0200005e->00000064 Xeon Scalable
SKX-D        M1       6-55-4/b7 0200005e->00000064 Xeon D-21xx
CLX-SP       B1       6-55-7/bf 05000021->0500002b Xeon Scalable Gen2

Comment 3 Jeff Bastian 2019-10-01 19:34:20 UTC
I verified that microcode_ctl-2.1-54.el7 is working with RHEL-7.8 Beta on 5 different Intel systems, including 2 from the list in comment 1.

https://beaker.engineering.redhat.com/jobs/3817232

Example from the Skylake system:

:::::::::::::::
:: Host Info ::
:::::::::::::::

[root@dell-per740-03 ~]# uname -r
3.10.0-1099.el7.x86_64

[root@dell-per740-03 ~]# lscpu | egrep -i -e family -e model -e stepping
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
Stepping:              4


::::::::::::
:: Before ::
::::::::::::

[root@dell-per740-03 ~]# rpm -q microcode_ctl
microcode_ctl-2.1-53.el7.x86_64

[root@dell-per740-03 ~]# cat /sys/devices/system/cpu/cpu0/microcode/version
0x200005e

[root@dell-per740-03 ~]# journalctl -b0 -o short-monotonic | sed "s/$(hostname) //" | grep -i microcode
[    0.000000] kernel: microcode: microcode updated early to revision 0x200005e, date = 2019-04-02
[    4.898183] kernel: microcode: sig=0x50654, pf=0x80, revision=0x200005e
[    4.901411] kernel: microcode: Microcode Update Driver: v2.01 <tigran.co.uk>, Peter Oruba
[   13.278145] systemd[1]: Starting Load CPU microcode update...
[   13.326230] systemd[1]: Started Load CPU microcode update.


:::::::::::
:: After ::
:::::::::::

[root@dell-per740-03 ~]# rpm -q microcode_ctl
microcode_ctl-2.1-54.el7.x86_64

[root@dell-per740-03 ~]# cat /sys/devices/system/cpu/cpu0/microcode/version
0x2000064

[root@dell-per740-03 ~]# journalctl -b0 -o short-monotonic | sed "s/$(hostname) //" | grep -i microcode
[    0.000000] kernel: microcode: microcode updated early to revision 0x2000064, date = 2019-07-31
[    5.005954] kernel: microcode: sig=0x50654, pf=0x80, revision=0x2000064
[    5.012107] kernel: microcode: Microcode Update Driver: v2.01 <tigran.co.uk>, Peter Oruba
[   14.063704] systemd[1]: Starting Load CPU microcode update...
[   14.108448] systemd[1]: Started Load CPU microcode update.

Comment 4 Jeff Bastian 2019-10-01 19:37:02 UTC
(In reply to Eugene Syromiatnikov from comment #1)
> SKX-SP       H0/M0/U0 6-55-4/b7 0200005e->00000064 Xeon Scalable
> SKX-D        M1       6-55-4/b7 0200005e->00000064 Xeon D-21xx
                                             ^
                                             ^
I think a bit got flipped here:              ^

That should be 02000064 based on my testing in comment 3, and also revision 00000064 would be a major downgrade.

Comment 5 Eugene Syromiatnikov 2019-10-01 21:17:58 UTC
Yes, that's correct, 00000064 seems to be a type in releasenotes.

Comment 6 Eugene Syromiatnikov 2019-10-01 23:52:49 UTC
(In reply to Jeff Bastian from comment #4)
> (In reply to Eugene Syromiatnikov from comment #1)
> > SKX-SP       H0/M0/U0 6-55-4/b7 0200005e->00000064 Xeon Scalable
> > SKX-D        M1       6-55-4/b7 0200005e->00000064 Xeon D-21xx
>                                              ^
>                                              ^
> I think a bit got flipped here:              ^
> 
> That should be 02000064 based on my testing in comment 3, and also revision
> 00000064 would be a major downgrade.

releasenote has been fixed upstream in [1].

[1] https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/commit/fd3eb05bb3b8a372eab06f4a286ae701b2e323bb

Comment 13 Jeff Bastian 2019-10-09 15:46:14 UTC
Re-tested and verified for the Sandy Bridge EP update (see bug 1758382).  This update installs microcode revision 0x714 by default, but 0x718 is available with a "force" file to override the blacklist.

::::::::::::::
:: Defaults ::
::::::::::::::

[root@hpe-z420-01 ~]# rpm -q microcode_ctl
microcode_ctl-2.1-55.el7.x86_64

[root@hpe-z420-01 ~]# lscpu | egrep -i -e family -e model -e stepping
CPU family:            6
Model:                 45
Model name:            Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Stepping:              7

[root@hpe-z420-01 ~]# uname -r
3.10.0-1099.el7.x86_64

[root@hpe-z420-01 ~]# cat /sys/devices/system/cpu/cpu0/microcode/version
0x714

[root@hpe-z420-01 ~]# journalctl -b0 -o short-monotonic | sed "s/$(hostname) //" | grep -i microcode | grep -v dracut
[    0.167711] kernel: MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    2.590618] kernel: microcode: sig=0x206d7, pf=0x1, revision=0x714
[    2.596195] kernel: microcode: Microcode Update Driver: v2.01 <tigran.co.uk>, Peter Oruba
[   14.971540] systemd[1]: Starting Load CPU microcode update...
[   16.056451] systemd[1]: Started Load CPU microcode update.
[   80.763916] restraintd[3038]: mds                Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable

[root@hpe-z420-01 ~]# cd /sys/devices/system/cpu/vulnerabilities

[root@hpe-z420-01 vulnerabilities]# grep . * | sed 's/:/^/' | column -t -s^
l1tf               Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
mds                Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
meltdown           Mitigation: PTI
spec_store_bypass  Mitigation: Speculative Store Bypass disabled via prctl and seccomp
spectre_v1         Mitigation: Load fences, usercopy/swapgs barriers and __user pointer sanitization
spectre_v2         Mitigation: Full retpoline, IBPB

:::::::::::::::::::::
:: With force file ::
:::::::::::::::::::::

[root@hpe-z420-01 ~]# install -D /dev/null /etc/microcode_ctl/ucode_with_caveats/force-intel-06-2d-07

[root@hpe-z420-01 ~]# dracut -f --early-microcode

[root@hpe-z420-01 ~]# reboot
...

[root@hpe-z420-01 ~]# cat /sys/devices/system/cpu/cpu0/microcode/version
0x718

[root@hpe-z420-01 ~]# journalctl -b0 -o short-monotonic | sed "s/$(hostname) //" | grep -i microcode | grep -v dracut
[    0.000000] kernel: microcode: microcode updated early to revision 0x718, date = 2019-05-21
[    2.589668] kernel: microcode: sig=0x206d7, pf=0x1, revision=0x718
[    2.595233] kernel: microcode: Microcode Update Driver: v2.01 <tigran.co.uk>, Peter Oruba
[   14.019966] systemd[1]: Starting Load CPU microcode update...
[   14.954415] systemd[1]: Started Load CPU microcode update.

[root@hpe-z420-01 ~]# cd /sys/devices/system/cpu/vulnerabilities

[root@hpe-z420-01 vulnerabilities]# grep . * | sed 's/:/^/' | column -t -s^
l1tf               Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
mds                Mitigation: Clear CPU buffers; SMT vulnerable
meltdown           Mitigation: PTI
spec_store_bypass  Mitigation: Speculative Store Bypass disabled via prctl and seccomp
spectre_v1         Mitigation: Load fences, usercopy/swapgs barriers and __user pointer sanitization
spectre_v2         Mitigation: Full retpoline, IBPB

Comment 14 Sandro Bonazzola 2020-01-14 09:06:47 UTC
Looks like the microcode_ctl released in RHEL 7.7 update broke RHV by changing MDS type on come CPUs and the fix for this bug is solving it.
Can we get this update in 7.7 as async?

Comment 15 Eugene Syromiatnikov 2020-01-14 11:13:42 UTC
(In reply to Sandro Bonazzola from comment #14)
> Looks like the microcode_ctl released in RHEL 7.7 update broke RHV by
> changing MDS type on come CPUs

What are you referring to? Is it related to bug 1710445? If this is the case, then it is covered by the respective KB articles[1][2].

> Can we get this update in 7.7 as async?

This specific update has been backported to 7.7.z via bug 1758572 (microcode_ctl-2.1-53.1.el7_7), as noted in comment 12.

[1] https://access.redhat.com/solutions/4593951
[2] https://access.redhat.com/solutions/4393691

Comment 16 Sandro Bonazzola 2020-02-25 12:22:24 UTC
microcode_ctl-2.1-53.1.el7_7 is older than microcode_ctl-2.1-53.7 which seems to be affected by  "CPU feature MDS is no longer presented" (bug #1784906) which seems not reproducible in microcode_ctl-2.1-61

Comment 17 Eugene Syromiatnikov 2020-02-25 13:12:51 UTC
Please refer to the KB articles mentioned in comment 15.

Comment 19 errata-xmlrpc 2020-03-31 20:08:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:1166