Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1367792

Summary:	After kernel update, zipl on s390x defaults to booting "_with_debugging" kernel
Product:	Red Hat Enterprise Linux 7	Reporter:	Andrew Price <anprice>
Component:	grubby	Assignee:	Peter Jones <pjones>
Status:	CLOSED WONTFIX	QA Contact:	Release Test Team <release-test-team-automation>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	7.3	CC:	anprice, dhorak, herrold, jonsulman, jstodola, mueller, radoslaw.piliszek, sbueno, schoudha
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-12-15 07:44:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Andrew Price 2016-08-17 13:28:13 UTC

Description of problem:

Some time between RHEL7.2 and current RHEL7.3 candidate, updating the kernel package on an s390x system, that boots with zipl, started switching the default from booting the normal boot menu entry to booting the _with_debugging menu entry.

Version-Release number of selected component (if applicable):

grubby-8.28-18.el7.s390x

How reproducible:

100%

Steps to Reproduce:
1. Install RHEL7.3 development on s390x virt
2. Make sure the default boot entry in /etc/zipl.conf is not a debugging entry
3. Update the kernel package
4. Observe that the default boot entry has become a _with_debugging entry

Actual results:

# zipl
Using config file '/etc/zipl.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section '3.10.0-489.el7.s390x'
Adding #2: IPL section '3.10.0-489.el7.s390x_with_debugging' (default)
Adding #3: IPL section '3.10.0-481.el7.s390x'
Adding #4: IPL section '3.10.0-481.el7.s390x_with_debugging'
Adding #5: IPL section '3.10.0-462.el7.s390x'
Preparing boot device: dasda (0100).
Done.

Expected results:

# zipl
Using config file '/etc/zipl.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section '3.10.0-489.el7.s390x' (default)
Adding #2: IPL section '3.10.0-489.el7.s390x_with_debugging'
Adding #3: IPL section '3.10.0-481.el7.s390x'
Adding #4: IPL section '3.10.0-481.el7.s390x_with_debugging'
Adding #5: IPL section '3.10.0-462.el7.s390x'
Preparing boot device: dasda (0100).
Done.


Additional info:

How my zipl.conf looks after the kernel update:

[defaultboot]
defaultauto
prompt=1
timeout=5
default=3.10.0-489.el7.s390x_with_debugging
target=/boot
[3.10.0-489.el7.s390x]
        image=/boot/vmlinuz-3.10.0-489.el7.s390x
        parameters="crashkernel=auto rd.lvm.lv=rhel_qe-c02-m02/root rd.dasd=0.0.0100 undefined root=/dev/mapper/rhel_qe--c02--m02-root rd.lvm.lv=rhel_qe-c02-m02/swap rd.dasd=0.0.0101 LANG=en_US.UTF-8"
        ramdisk=/boot/initramfs-3.10.0-489.el7.s390x.img
[3.10.0-489.el7.s390x_with_debugging]
        image=/boot/vmlinuz-3.10.0-489.el7.s390x
        parameters="crashkernel=auto rd.lvm.lv=rhel_qe-c02-m02/root rd.dasd=0.0.0100 undefined root=/dev/mapper/rhel_qe--c02--m02-root rd.lvm.lv=rhel_qe-c02-m02/swap rd.dasd=0.0.0101 LANG=en_US.UTF-8 systemd.log_level=debug systemd.log_target=kmsg"
        ramdisk=/boot/initramfs-3.10.0-489.el7.s390x.img
[3.10.0-481.el7.s390x]
        image=/boot/vmlinuz-3.10.0-481.el7.s390x
        parameters="crashkernel=auto rd.lvm.lv=rhel_qe-c02-m02/root rd.dasd=0.0.0100 undefined root=/dev/mapper/rhel_qe--c02--m02-root rd.lvm.lv=rhel_qe-c02-m02/swap rd.dasd=0.0.0101 LANG=en_US.UTF-8"
        ramdisk=/boot/initramfs-3.10.0-481.el7.s390x.img
[3.10.0-481.el7.s390x_with_debugging]
        image=/boot/vmlinuz-3.10.0-481.el7.s390x
        parameters="crashkernel=auto rd.lvm.lv=rhel_qe-c02-m02/root rd.dasd=0.0.0100 undefined root=/dev/mapper/rhel_qe--c02--m02-root rd.lvm.lv=rhel_qe-c02-m02/swap rd.dasd=0.0.0101 LANG=en_US.UTF-8 systemd.log_level=debug systemd.log_target=kmsg"
        ramdisk=/boot/initramfs-3.10.0-481.el7.s390x.img
[3.10.0-462.el7.s390x]
        image=/boot/vmlinuz-3.10.0-462.el7.s390x
        parameters="crashkernel=auto rd.lvm.lv=rhel_qe-c02-m02/root rd.dasd=0.0.0100 undefined root=/dev/mapper/rhel_qe--c02--m02-root rd.lvm.lv=rhel_qe-c02-m02/swap rd.dasd=0.0.0101 LANG=en_US.UTF-8"
        ramdisk=/boot/initramfs-3.10.0-462.el7.s390x.img

Comment 1 Peter Jones 2016-09-08 17:16:50 UTC

Are you sure this was done with grubby-8.28-18.el7.s390x , and that wasn't merely the version installed after the rpm update was finished?  To me this looks like something I'd expect -17.el7 to do, but -18.el7 should have fixed.

That is, if you upgrade grubby in its own transaction before updating anything else, does the error still occur?

Comment 2 Jan Stodola 2016-09-09 13:36:07 UTC

Reproduced with grubby-8.28-18.el7.s390x (compose RHEL-7.3-20160901.1):

[root@rtt7 ~]# zipl
Using config file '/etc/zipl.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section 'linux' (default)
Adding #2: IPL section 'linux-0-rescue-28b817321aa14230b612ddd82169abe9'
Preparing boot device: dasda (3627).
Done.
[root@rtt7 ~]# echo "MAKEDEBUG=yes" >> /etc/sysconfig/kernel
[root@rtt7 ~]# yum install ./kernel-3.10.0-501.el7.s390x.rpm
[root@rtt7 ~]# zipl
Using config file '/etc/zipl.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section '3.10.0-501.el7.s390x'
Adding #2: IPL section '3.10.0-501.el7.s390x_with_debugging' (default)
Adding #3: IPL section 'linux'
Adding #4: IPL section 'linux-0-rescue-28b817321aa14230b612ddd82169abe9'
Preparing boot device: dasda (3627).
Done.
[root@rtt7 ~]# rpm -q grubby
grubby-8.28-18.el7.s390x
[root@rtt7 ~]#

Comment 3 Jan Stodola 2016-09-09 13:43:33 UTC

A note - to reproduce it, it is necessary to add MAKEDEBUG=yes to /etc/sysconfig/kernel. This option is not there by default, so without this change this bug problem won't appear.

Comment 4 Samantha N. Bueno 2016-09-09 21:04:09 UTC

(In reply to Jan Stodola from comment #3)
> A note - to reproduce it, it is necessary to add MAKEDEBUG=yes to
> /etc/sysconfig/kernel. This option is not there by default, so without this
> change this bug problem won't appear.

Based on this information -- which indicates to me that default behavior works fine -- this is not a blocker, so I'm deferring this to 7.4 planning.

Clearing pjones' needinfo request as well since jstodola tested it with -18 version.

Comment 5 Jan Stodola 2016-09-16 09:05:01 UTC

BTW, it's not only on s390x with zipl, but also on x86_64, see bug 1285601#c5.

Comment 6 Thomas Mueller 2016-11-09 15:45:26 UTC

I have this issue with x86_64 and grub2 since upgrade to RHEL 7.3.

The /boot/grub2/grub.cfg entries are now in correct order (first entry: normal boot, second entry: with debugging, RHEL 7.2 had this in reverse) but grub2-editenv list show saved_entry with `... with debugging`.

Comment 7 Thomas Mueller 2016-11-09 15:51:48 UTC

(In reply to Thomas Mueller from comment #6)
> I have this issue with x86_64 and grub2 since upgrade to RHEL 7.3.
> 
> The /boot/grub2/grub.cfg entries are now in correct order (first entry:
> normal boot, second entry: with debugging, RHEL 7.2 had this in reverse) but
> grub2-editenv list show saved_entry with `... with debugging`.

/etc/syconfig/kernel has MAKEDEBUG=yes altough i never added this. interestingly 

rm /etc/sysconfig/kernel
rpm --erase --nodeps grubby
yum install grubby

does not restore the file. It seems it is added somewhere while install?

Comment 8 Radosław Piliszek 2017-02-16 08:56:53 UTC

The same happens on x86_64 with GRUB.

New kernels default to debug when MAKEDEBUG=yes which in turn got installed with some update but did not get reverted.

The workaround is to remove MAKEDEBUG=yes from /etc/sysconfig/kernel and set correct default manually. Next kernel update will not break it unless MAKEDEBUG=yes appears magically again.

Please fix new-kernel-pkg not to default to debug.

Comment 9 Radosław Piliszek 2017-02-16 09:12:59 UTC

More info:
Documentation states:
'MAKEDEBUG specifies if new-kernel-pkg should create non-default "debug" entries for new kernels.'
So they should be non-default by design, not default. This is clearly a nasty bug.

Comment 12 RHEL Program Management 2020-12-15 07:44:39 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.