Bug 1574593

Summary:	[RHEL-6] Include updated ucode for Broadwell EP/EX
Product:	Red Hat Enterprise Linux 6	Reporter:	Stanislav Kozina <skozina>
Component:	microcode_ctl	Assignee:	Eugene Syromiatnikov <esyr>
Status:	CLOSED ERRATA	QA Contact:	Jeff Bastian <jbastian>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	6.10	CC:	cww, dsoman, jbastian, lwang, rasibley, skozina, snagar, toneata
Target Milestone:	rc	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	microcode_ctl-1.17-33.3.el6_10	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1576323 1576324 1576325 1576326 1576327 (view as bug list)		Environment:
Last Closed:	2018-07-31 12:01:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1574592
Bug Blocks:	1576323, 1576324, 1576325, 1576326, 1576327

Description Stanislav Kozina 2018-05-03 15:35:12 UTC

The updated ucode needs to be delivered for Broadwell EP/EX CPU models to mitigate Spectre vulnerability.
Broadwell EP/EX CPUs need updated microcode update sequence so this is blocker by bz1574592.

Comment 9 Jeff Bastian 2018-07-20 17:43:31 UTC

Verified on RHEL-6.10.z on an Intel Sandy Bridge, Broadwell, and Skylake systems.  Posting test results in one comment per system.

Starting with Broadwell-EP since it's the subject of this BZ.

:::::::::::::::
:: Host Info ::
:::::::::::::::

[root@dell-per730-02 ~]# hostname
dell-per730-02.khw.lab.eng.bos.redhat.com

[root@dell-per730-02 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Workstation release 6.10 (Santiago)

[root@dell-per730-02 ~]# egrep -m4 'family|model|stepping' /proc/cpuinfo
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
stepping	: 1

::::::::::::
:: Before ::
::::::::::::

[root@dell-per730-02 ~]# uname -r
2.6.32-754.el6.x86_64

[root@dell-per730-02 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-32.el6.x86_64

[root@dell-per730-02 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
microcode: CPU1 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
                              ...8<...snip...8<...
microcode: CPU39 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba

[root@dell-per730-02 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full retpoline

[root@dell-per730-02 ~]# grep -m1 flags /proc/cpuinfo > flags.before

:::::::::::
:: After ::
:::::::::::

[root@dell-per730-02 ~]# yum -y update
...
Installing:
 kernel              x86_64     2.6.32-754.2.1.el6      kernel             32 M
 kernel-devel        x86_64     2.6.32-754.2.1.el6      kernel             11 M
Updating:
 kernel-firmware     noarch     2.6.32-754.2.1.el6      kernel             29 M
 kernel-headers      x86_64     2.6.32-754.2.1.el6      kernel            4.5 M
 microcode_ctl       x86_64     2:1.17-33.3.el6_10      microcode_ctl     1.8 M
...

[root@dell-per730-02 ~]# reboot
...


[root@dell-per730-02 ~]# uname -r
2.6.32-754.2.1.el6.x86_64

[root@dell-per730-02 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-33.3.el6_10.x86_64

[root@dell-per730-02 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
microcode: CPU1 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
                              ...8<...snip...8<...
microcode: CPU0 updated to revision 0xb00002e, date = 2018-04-19 
microcode: CPU1 updated to revision 0xb00002e, date = 2018-04-19 
microcode: CPU2 updated to revision 0xb00002e, date = 2018-04-19 
microcode: CPU3 updated to revision 0xb00002e, date = 2018-04-19 
                              ...8<...snip...8<...
microcode: CPU39 updated to revision 0xb00002e, date = 2018-04-19 

[root@dell-per730-02 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full retpoline, IBPB

[root@dell-per730-02 ~]# grep -m1 flags /proc/cpuinfo > flags.after

[root@dell-per730-02 ~]# diff -U0 <(sed 's/\s/\n/g' flags.before | sort) \
                                  <(sed 's/\s/\n/g' flags.after | sort)
--- /dev/fd/63	2018-07-20 13:26:23.997406670 -0400
+++ /dev/fd/62	2018-07-20 13:26:23.997406670 -0400
@@ -30,0 +31 @@
+eagerfpu
@@ -85,0 +87 @@
+ssbd

:::::::::::::
:: Results ::
:::::::::::::

The microcode was successfully updated from 0xb00002a to 0xb00002e, and it added the new CPU flag ssbd which enables the kernel's SSBD mitigations.

Comment 10 Jeff Bastian 2018-07-20 17:44:28 UTC

Sandy Bridge

:::::::::::::::
:: Host Info ::
:::::::::::::::

[root@intel-lizardhead-01 ~]# hostname
intel-lizardhead-01.lab.bos.redhat.com

[root@intel-lizardhead-01 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Workstation release 6.10 (Santiago)

[root@intel-lizardhead-01 ~]# egrep -m4 'family|model|stepping' /proc/cpuinfo
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-4650 0 @ 2.70GHz
stepping	: 7

::::::::::::
:: Before ::
::::::::::::

[root@intel-lizardhead-01 ~]# uname -r
2.6.32-754.el6.x86_64

[root@intel-lizardhead-01 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-32.el6.x86_64

[root@intel-lizardhead-01 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU1 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU2 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU3 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
                              ...8<...snip...8<...
microcode: CPU63 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
microcode: CPU0 updated to revision 0x713, date = 2018-01-26 
microcode: CPU1 updated to revision 0x713, date = 2018-01-26 
microcode: CPU2 updated to revision 0x713, date = 2018-01-26 
                              ...8<...snip...8<...
microcode: CPU63 updated to revision 0x713, date = 2018-01-26 

[root@intel-lizardhead-01 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full retpoline

[root@intel-lizardhead-01 ~]# grep -m1 flags /proc/cpuinfo > flags.before

:::::::::::
:: After ::
:::::::::::

[root@intel-lizardhead-01 ~]# yum -y update
...
Installing:
 kernel              x86_64     2.6.32-754.2.1.el6      kernel             32 M
 kernel-devel        x86_64     2.6.32-754.2.1.el6      kernel             11 M
Updating:
 kernel-firmware     noarch     2.6.32-754.2.1.el6      kernel             29 M
 kernel-headers      x86_64     2.6.32-754.2.1.el6      kernel            4.5 M
 microcode_ctl       x86_64     2:1.17-33.3.el6_10      microcode_ctl     1.8 M
...

[root@intel-lizardhead-01 ~]# reboot
...


[root@intel-lizardhead-01 ~]# uname -r
2.6.32-754.2.1.el6.x86_64

[root@intel-lizardhead-01 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-33.3.el6_10.x86_64

[root@intel-lizardhead-01 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU1 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU2 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
                              ...8<...snip...8<...
microcode: CPU63 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
microcode: CPU0 updated to revision 0x714, date = 2018-05-08 
microcode: CPU1 updated to revision 0x714, date = 2018-05-08 
microcode: CPU2 updated to revision 0x714, date = 2018-05-08 
                              ...8<...snip...8<...
microcode: CPU63 updated to revision 0x714, date = 2018-05-08 

[root@intel-lizardhead-01 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full retpoline, IBPB

[root@intel-lizardhead-01 ~]# grep -m1 flags /proc/cpuinfo > flags.after

[root@intel-lizardhead-01 ~]# diff -U0 <(sed 's/\s/\n/g' flags.before | sort) \
                                       <(sed 's/\s/\n/g' flags.after | sort)
--- /dev/fd/63	2018-07-20 13:37:46.664798175 -0400
+++ /dev/fd/62	2018-07-20 13:37:46.667798073 -0400
@@ -21,0 +22 @@
+eagerfpu
@@ -64,0 +66 @@
+ssbd

:::::::::::::
:: Results ::
:::::::::::::

The microcode was successfully updated from 0x70d to 0x714, and it added the new CPU flag ssbd which enables the kernel's SSBD mitigations.

Comment 11 Jeff Bastian 2018-07-20 17:45:21 UTC

Skylake

:::::::::::::::
:: Host Info ::
:::::::::::::::

[root@dell-pet7920-02 ~]# hostname
dell-pet7920-02.rhts.eng.bos.redhat.com

[root@dell-pet7920-02 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Workstation release 6.10 (Santiago)

[root@dell-pet7920-02 ~]# egrep -m4 'family|model|stepping' /proc/cpuinfo
cpu family	: 6
model		: 85
model name	: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
stepping	: 4

::::::::::::
:: Before ::
::::::::::::

[root@dell-pet7920-02 ~]# uname -r
2.6.32-754.el6.x86_64

[root@dell-pet7920-02 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-32.el6.x86_64

[root@dell-pet7920-02 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU1 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU2 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
                              ...8<...snip...8<...
microcode: CPU63 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
microcode: CPU0 updated to revision 0x2000043, date = 2018-01-26 
microcode: CPU1 updated to revision 0x2000043, date = 2018-01-26 
microcode: CPU2 updated to revision 0x2000043, date = 2018-01-26 
                              ...8<...snip...8<...
microcode: CPU63 updated to revision 0x2000043, date = 2018-01-26 

[root@dell-pet7920-02 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: IBRS (kernel)

[root@dell-pet7920-02 ~]# grep -m1 flags /proc/cpuinfo > flags.before

:::::::::::
:: After ::
:::::::::::

[root@dell-pet7920-02 ~]# yum -y update
...
Installing:
 kernel              x86_64     2.6.32-754.2.1.el6      kernel             32 M
 kernel-devel        x86_64     2.6.32-754.2.1.el6      kernel             11 M
Updating:
 kernel-firmware     noarch     2.6.32-754.2.1.el6      kernel             29 M
 kernel-headers      x86_64     2.6.32-754.2.1.el6      kernel            4.5 M
 microcode_ctl       x86_64     2:1.17-33.3.el6_10      microcode_ctl     1.8 M
...

[root@dell-pet7920-02 ~]# reboot
...


[root@dell-pet7920-02 ~]# uname -r
2.6.32-754.2.1.el6.x86_64

[root@dell-pet7920-02 ~]# rpm -q microcode_ctl
microcode_ctl-1.17-33.3.el6_10.x86_64

[root@dell-pet7920-02 ~]# grep -i microcode /var/log/dmesg
microcode: CPU0 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU1 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU2 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
                              ...8<...snip...8<...
microcode: CPU63 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
microcode: CPU0 updated to revision 0x200004d, date = 2018-05-15 
microcode: CPU1 updated to revision 0x200004d, date = 2018-05-15 
microcode: CPU2 updated to revision 0x200004d, date = 2018-05-15 
                              ...8<...snip...8<...
microcode: CPU63 updated to revision 0x200004d, date = 2018-05-15 

[root@dell-pet7920-02 ~]# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: IBRS (kernel), IBPB

[root@dell-pet7920-02 ~]# grep -m1 flags /proc/cpuinfo > flags.after

[root@dell-pet7920-02 ~]# diff -U0 <(sed 's/\s/\n/g' flags.before | sort) <(sed 's/\s/\n/g' flags.after | sort)
--- /dev/fd/63	2018-07-20 13:39:46.271478092 -0400
+++ /dev/fd/62	2018-07-20 13:39:46.265477810 -0400
@@ -32,0 +33 @@
+eagerfpu
@@ -91,0 +93 @@
+ssbd

:::::::::::::
:: Results ::
:::::::::::::

The microcode was successfully updated from 0x2000014 to 0x200004d, and it added the new CPU flag ssbd which enables the kernel's SSBD mitigations.

Comment 12 Jeff Bastian 2018-07-20 17:59:00 UTC

Finally, one last test with Broadwell-EP: if the system boots the RHEL-6.10 GA kernel, it should _not_ update the microcode since the system may freeze.  The new checks in /usr/libexec/microcode_ctl/check_kver will block the microcode update on older kernels.


[root@dell-per730-02 ~]# grubby --set-default /boot/vmlinuz-2.6.32-754.el6.x86_64

[root@dell-per730-02 ~]# reboot
...

[root@dell-per730-02 ~]# uname -r
2.6.32-754.el6.x86_64

[root@dell-per730-02 ~]# grep microcode /var/log/dmesg
microcode: CPU0 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
microcode: CPU1 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
microcode: CPU2 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01
                              ...8<...snip...8<...
microcode: CPU39 sig=0x406f1, pf=0x1, revision=0xb00002a
platform microcode: firmware: requesting intel-ucode/06-4f-01

[root@dell-per730-02 ~]# 

                    ========================================

However, the Sandy Bridge and Skylake systems are safe to update even on the older kernel, and the update still works:

::::::::::::::::::
:: Sandy Bridge ::
::::::::::::::::::

[root@intel-lizardhead-01 ~]# grubby --set-default /boot/vmlinuz-2.6.32-754.el6.x86_64

[root@intel-lizardhead-01 ~]# reboot
...

[root@intel-lizardhead-01 ~]# uname -r
2.6.32-754.el6.x86_64

[root@intel-lizardhead-01 ~]# grep microcode /var/log/dmesg
microcode: CPU0 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU1 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
microcode: CPU2 sig=0x206d7, pf=0x40, revision=0x70d
platform microcode: firmware: requesting intel-ucode/06-2d-07
                              ...8<...snip...8<...
microcode: CPU0 updated to revision 0x714, date = 2018-05-08 
microcode: CPU1 updated to revision 0x714, date = 2018-05-08 
microcode: CPU2 updated to revision 0x714, date = 2018-05-08 
                              ...8<...snip...8<...

[root@intel-lizardhead-01 ~]# 

:::::::::::::
:: Skylake ::
:::::::::::::

[root@dell-pet7920-02 ~]# grubby --set-default /boot/vmlinuz-2.6.32-754.el6.x86_64

[root@dell-pet7920-02 ~]# reboot
...

[root@dell-pet7920-02 ~]# uname -r
2.6.32-754.el6.x86_64

[root@dell-pet7920-02 ~]# grep microcode /var/log/dmesg
microcode: CPU0 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU1 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
microcode: CPU2 sig=0x50654, pf=0x80, revision=0x2000014
platform microcode: firmware: requesting intel-ucode/06-55-04
                              ...8<...snip...8<...
microcode: CPU0 updated to revision 0x200004d, date = 2018-05-15 
microcode: CPU1 updated to revision 0x200004d, date = 2018-05-15 
microcode: CPU2 updated to revision 0x200004d, date = 2018-05-15 
                              ...8<...snip...8<...

[root@dell-pet7920-02 ~]# 

                    ========================================

With the older kernel, the microcode was _not_ updated on Broadwell-EP (which is good), but it was successfully updated on Sandy Bridge and Skylake.

Comment 14 errata-xmlrpc 2018-07-31 12:01:26 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2300