Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1721815

Summary:	grub2 can't choose the correct kernel in ppc64le
Product:	Red Hat Enterprise Linux 8	Reporter:	Chunyu Hu <chuhu>
Component:	grub2	Assignee:	Bootloader engineering team <bootloader-eng-team>
Status:	CLOSED ERRATA	QA Contact:	Release Test Team <release-test-team-automation>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.1	CC:	fmartine, mbanas, pzatko
Target Milestone:	rc	Flags:	pm-rhel: mirror+
Target Release:	8.0
Hardware:	ppc64le
OS:	Linux
Whiteboard:
Fixed In Version:	grub2-2.02-75.el8	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-11-05 22:24:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chunyu Hu 2019-06-19 06:20:03 UTC

Description of problem:

This is stable reproducible with ppc64le, please resolve this in rhel-8.1. It's blocking users testing or doing kernel switches. 

[root@ibm-p9b-03 ~]# grubby --info DEFAULT
index=1
kernel="/boot/vmlinuz-4.18.0-100.el8.ppc64le"
args="ro crashkernel=auto rd.lvm.lv=rhel_ibm-p9b-03/root rd.lvm.lv=rhel_ibm-p9b-03/swap $tuned_params nohz=on nohz_full=0-10 rcu_nocbs=0-10 irqaffinity=159 nowatchdog nmi_watchdog=0 nosoftlockup rcu_nocb_poll"
root="/dev/mapper/rhel_ibm--p9b--03-root"
initrd="/boot/initramfs-4.18.0-100.el8.ppc64le.img $tuned_initrd"
title="Red Hat Enterprise Linux (4.18.0-100.el8.ppc64le) 8.1 (Ootpa)"
id="28af637bf44e415aaf2c03bfeee19d82-4.18.0-100.el8.ppc64le"
[root@ibm-p9b-03 ~]# rhts-reboot 

After - reboot:

[root@ibm-p9b-03 ~]# uname -r
4.18.0-101.el8.ppc64le
[root@ibm-p9b-03 ~]# rpm -q grubby
grubby-8.40-37.el8.ppc64le
[root@ibm-p9b-03 ~]# grubby --info DEFAULT
index=1
kernel="/boot/vmlinuz-4.18.0-100.el8.ppc64le"
args="ro crashkernel=auto rd.lvm.lv=rhel_ibm-p9b-03/root rd.lvm.lv=rhel_ibm-p9b-03/swap $tuned_params nohz=on nohz_full=0-10 rcu_nocbs=0-10 irqaffinity=159 nowatchdog nmi_watchdog=0 nosoftlockup rcu_nocb_poll"
root="/dev/mapper/rhel_ibm--p9b--03-root"
initrd="/boot/initramfs-4.18.0-100.el8.ppc64le.img $tuned_initrd"
title="Red Hat Enterprise Linux (4.18.0-100.el8.ppc64le) 8.1 (Ootpa)"
id="28af637bf44e415aaf2c03bfeee19d82-4.18.0-100.el8.ppc64le"

[root@ibm-p9b-03 ~]# cat /boot/grub2/grubenv 
# GRUB Environment Block
saved_entry=28af637bf44e415aaf2c03bfeee19d82-4.18.0-100.el8.ppc64le
kernelopts=root=/dev/mapper/rhel_ibm--p9b--03-root ro crashkernel=auto rd.lvm.lv=rhel_ibm-p9b-03/root rd.lvm.lv=rhel_ibm-p9b-03/swap 
boot_success=0
boot_indeterminate=
##########################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################[
Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1.Choose ppc64le with above setup.
2. install two kernels.
3.  grubby --set-default lower version kernel.

Actual results:
booted to newer kernel, version not matching the grub2 config. 

Expected results:


Additional info:

Comment 1 Chunyu Hu 2019-06-19 06:21:57 UTC

[root@ibm-p9b-03 ~]# rpm -q kernel
kernel-4.18.0-100.el8.ppc64le
kernel-4.18.0-101.el8.ppc64le
[root@ibm-p9b-03 ~]# ls /boot/vmlinuz-* -hl
-rwxr-xr-x. 1 root root 23M Jun 17 05:01 /boot/vmlinuz-0-rescue-28af637bf44e415aaf2c03bfeee19d82
-rwxr-xr-x. 1 root root 23M May 31 22:48 /boot/vmlinuz-4.18.0-100.el8.ppc64le
-rwxr-xr-x. 1 root root 23M Jun  6 07:31 /boot/vmlinuz-4.18.0-101.el8.ppc64le

Comment 2 Javier Martinez Canillas 2019-06-19 07:16:49 UTC

I'll try to reproduce it. Is this a machine with OPAL or OF ?

Comment 3 Chunyu Hu 2019-06-19 07:32:57 UTC

https://beaker.engineering.redhat.com/view/ibm-p9b-03.pnr.lab.eng.bos.redhat.com

Host Hypervisor 	(not virtualized)
Vendor 	IBM
Model 	5104-22C
Serial Number 	1302LTA
MAC Address 	0c:c4:7a:f4:1e:16
Memory 	524288 MB
NUMA Nodes 	2

[root@ibm-p9b-03 tracing]# lscpu 
Architecture:        ppc64le
Byte Order:          Little Endian
CPU(s):              160
On-line CPU(s) list: 0-159
Thread(s) per core:  4
Core(s) per socket:  20
Socket(s):           2
NUMA node(s):        2
Model:               2.2 (pvr 004e 1202)
Model name:          POWER9, altivec supported
CPU max MHz:         3800.0000
CPU min MHz:         2166.0000
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            10240K
NUMA node0 CPU(s):   0-79
NUMA node8 CPU(s):   80-159

[root@ibm-p9b-03 tracing]# lshw | grep -i opal -A 10
    capabilities: smp powernv opal
  *-core
       description: Motherboard
       physical id: 0
     *-cpu:0
          description: POWER9, altivec supported
          product: 02CY228
          physical id: 0
          bus info: cpu@0
          version: 2.2 (pvr 004e 1202)
          serial: YA1934558877

Comment 4 Javier Martinez Canillas 2019-06-19 13:37:16 UTC

This is a bug in the OPAL/Petitboot firmware. The problem is that Petitboot can only recognize the menuentry command --id option argument if this is separated by a '='. It does not support to separate the option and the argument with a space, even when this is the convention used by GRUB scripts when adding menu entries commands to the GRUB config file.

This can be easily tested with:

$ sed -i -s 's/--id /--id=/g' /boot/grub2/grub.cfg

I'll propose a fix for Petitboot to also support '--id foo' but we need to support current firmware versions, so as a workaround we can use '--id=foo' for the menu entries to allow Petitboot to correctly parse the argument.

Comment 13 errata-xmlrpc 2019-11-05 22:24:10 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3587