Bug 1651686 - with BLS enabled, not passing kernel options when adding new kernels
Summary: with BLS enabled, not passing kernel options when adding new kernels
Keywords:
Status: CLOSED DUPLICATE of bug 1690765
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: grub2
Version: 8.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: 8.0
Assignee: Javier Martinez Canillas
QA Contact: Release Test Team
URL:
Whiteboard:
: 1653566 1654322 (view as bug list)
Depends On: 1681759
Blocks: 1699081 1717075
TreeView+ depends on / blocked
 
Reported: 2018-11-20 15:06 UTC by Tom Tracy
Modified: 2019-10-24 09:34 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-24 09:34:37 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)

Description Tom Tracy 2018-11-20 15:06:34 UTC
Description of problem:
Before bls, we would add kernel options into /etc/default/grub in rhel7 and rhe8 and the options would be added /boot/grub2/grub.cfg. 

Now with the kernel options in /etc/default/grub, with BLS on, the only way to add kernel options is to add them manually to /boot/loader/entries/*.conf file


Version-Release number of selected component (if applicable):
grub2 2.02-59

How reproducible:
Every time


Steps to Reproduce:
1. Add kernel options to /etc/default/grub
2. Add  new kernel - 
3. Kernel options not added in /boot/loader/entries

Actual results: 
No kernel options were added


Expected results:
Kernel options should be added in the /boot/loader/entries/*.conf file


Additional info:
Following the google doc https://docs.google.com/document/d/19fhAi9FrOJnK_5KAK7v06bsztXeVmEknLJQEq3g3b2Q/edit#

tried adding kernel options here
grub2-editenv - set kernelopts=”rd.debug=1 rhgb ipv6.disable=1 intel_iommu=on iommu=pt”
But syntax is not correct and after trying to reboot, the system did not reboot 
as something was missing

Want to be able to set kernelopts so anytime I build the kernel, kernel options are passed and do not have to add manually

Comment 1 Javier Martinez Canillas 2018-11-20 15:37:11 UTC
(In reply to Tom Tracy from comment #0)
> Description of problem:
> Before bls, we would add kernel options into /etc/default/grub in rhel7 and
> rhe8 and the options would be added /boot/grub2/grub.cfg. 
> 
> Now with the kernel options in /etc/default/grub, with BLS on, the only way
> to add kernel options is to add them manually to /boot/loader/entries/*.conf
> file
> 

You can also use the grubby tool to edit the kernel options for just one entry, for example:

$ grubby --update-kernel /boot/vmlinuz-4.18.0-40.el8.x86_64 --args="rd.debug=1 rhgb ipv6.disable=1 intel_iommu=on iommu=pt"

# grubby --info /boot/vmlinuz-4.18.0-40.el8.x86_64 | grep args
args="ro crashkernel=auto resume=/dev/mapper/rhel_dell--r730--002--guest24-swap rd.lvm.lv=rhel_dell-r730-002-guest24/root rd.lvm.lv=rhel_dell-r730-002-guest24/swap console=ttyS0,115200 rd.debug=1 rhgb ipv6.disable=1 intel_iommu=pt"

Keep in mind though that grubby stores the modified kernel options in the BLS file. So changing $kernelopts won't have effect for an entry after it has been modified using grubby. 

> 
> Additional info:
> Following the google doc
> https://docs.google.com/document/d/
> 19fhAi9FrOJnK_5KAK7v06bsztXeVmEknLJQEq3g3b2Q/edit#
> 
> tried adding kernel options here
> grub2-editenv - set kernelopts=”rd.debug=1 rhgb ipv6.disable=1
> intel_iommu=on iommu=pt”
> But syntax is not correct and after trying to reboot, the system did not
> reboot 
> as something was missing
> 
> Want to be able to set kernelopts so anytime I build the kernel, kernel
> options are passed and do not have to add manually

That document isn't clear that grub2-editenv doesn't append but instead sets the value of the variable. So if you want to modify $kernelopts you need to do something like:

# kernelopts=$(grub2-editenv list | grep kernelopts | sed 's/kernelopts=//')
# grub2-editenv - set kernelopts="$kernelopts rd.debug=1 rhgb ipv6.disable=1 intel_iommu=on iommu=pt"

That would append the parameters as you want:

# grub2-editenv list | grep kernelopts
kernelopts=root=/dev/mapper/rhel_dell--r730--002--guest24-root ro crashkernel=auto resume=/dev/mapper/rhel_dell--r730--002--guest24-swap rd.lvm.lv=rhel_dell-r730-002-guest24/root rd.lvm.lv=rhel_dell-r730-002-guest24/swap console=ttyS0,115200  rd.debug=1 rhgb ipv6.disable=1 intel_iommu=on iommu=pt

Comment 2 Tom Tracy 2018-11-20 16:33:23 UTC
Ok so the google docs are not clear, I see that now. 

Your example above does work as I added a kernel after running grub2-editenv. I suggest the doc be updated so that people can add the needed kernel options especially in a kvm/RHV environment

Thanks
Tom

Comment 3 Tom Tracy 2018-11-20 21:01:15 UTC
If I want to remove kernel options how do I do that. Testing retpoline effects and can not seem to remove it from the kernel option. 

grub2-editenv list | grep kernelopts
kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root rd.lvm.lv=rhel18_perf18/swap  rd.debug=1 rhgb ipv6.disable=1 mem=6G spectre_v2=retpoline

[root@perf18 certification]# grub2-editenv - set kernelopts="$kernelopts"
[root@perf18 certification]# grub2-editenv list | grep kernelopts
kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root rd.lvm.lv=rhel18_perf18/swap  rd.debug=1 rhgb ipv6.disable=1 mem=6G spectre_v2=retpoline

Want to be able to enable/disable these flags but can not find any documentation on it

Thanks
Tom

Comment 4 Matthew Secaur 2018-11-21 18:37:29 UTC
(In reply to Tom Tracy from comment #3)
> If I want to remove kernel options how do I do that. Testing retpoline
> effects and can not seem to remove it from the kernel option. 
> 
> grub2-editenv list | grep kernelopts
> kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto
> resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root
> rd.lvm.lv=rhel18_perf18/swap  rd.debug=1 rhgb ipv6.disable=1 mem=6G
> spectre_v2=retpoline
> 
> [root@perf18 certification]# grub2-editenv - set kernelopts="$kernelopts"
> [root@perf18 certification]# grub2-editenv list | grep kernelopts
> kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto
> resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root
> rd.lvm.lv=rhel18_perf18/swap  rd.debug=1 rhgb ipv6.disable=1 mem=6G
> spectre_v2=retpoline
> 
> Want to be able to enable/disable these flags but can not find any
> documentation on it
> 
> Thanks
> Tom

Tom,

I think the easiest way is to simply extract the kernel opts from the current set (i.e. the ones you had previously enabled), and then manually remove them and run them through grub2-editenv again.

For example:

# grub2-editenv list | grep kernelopts
kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root rd.lvm.lv=rhel18_perf18/swap  rd.debug=1 rhgb ipv6.disable=1 mem=6G spectre_v2=retpoline

Now we want to remove, say, everything from "rd.debug=1" to the end:

# kernelopts="root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root rd.lvm.lv=rhel18_perf18/swap"

Finally, run grub2-editenv again (you can easily replace $kernelopts below with the actual options, instead):
# grub2-editenv - set kernelopts="$kernelopts"

# grub2-editenv list | grep kernelopts
kernelopts=root=/dev/mapper/rhel18_perf18-root ro crashkernel=auto resume=/dev/mapper/rhel18_perf18-swap rd.lvm.lv=rhel18_perf18/root rd.lvm.lv=rhel18_perf18/swap

Hope this is what you meant.
Matthew

Comment 5 Tom Tracy 2018-11-26 13:39:14 UTC
Matt
         This is exactly what I meant. With your examples above, I am getting a better understanding how this works. If there is "any" documentation I can pass along to my team (we deal we plenty of kernel parameter changes), that would be great

Tom

Comment 6 Javier Martinez Canillas 2018-11-28 14:30:12 UTC
*** Bug 1654322 has been marked as a duplicate of this bug. ***

Comment 7 Raghavendra Rao 2018-11-29 08:59:31 UTC
(In reply to Tom Tracy from comment #5)
> Matt
>          This is exactly what I meant. With your examples above, I am
> getting a better understanding how this works. If there is "any"
> documentation I can pass along to my team (we deal we plenty of kernel
> parameter changes), that would be great
> 
> Tom

Documentation which has most of the things covered: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8-beta/html/managing_monitoring_and_updating_the_kernel/configuring-kernel-command-line-parameters_managing-monitoring-and-updating-the-kernel#what-are-boot-entries_configuring-kernel-command-line-parameters

Moreover, rather than using commands I would prefer to directly use the conventional way of doing things in RHEL8:

Directly set 'kernelopts' variable present under /boot/grub2/grubenv according to your choice, save the file and simply reboot the system to get the kernel booted with the desired command-line parameters. In fact, all the grub-commands take reference from and work on this file itself.

I have mentioned the same in the following KB with some reference: https://access.redhat.com/solutions/3698411

Hope this helps.

-R.Rao

Comment 8 Javier Martinez Canillas 2018-11-29 11:52:13 UTC
*** Bug 1653566 has been marked as a duplicate of this bug. ***

Comment 11 David A. De Graaf 2019-06-03 21:25:25 UTC
I, too, find the default kernel options for F30 unacceptable.
Based on Matthew Secaur's advice in Comment 4, I wrote a little
script to fix them:
# cat /usr/local/bin/fixkerneloptions
# Fix kernel options,  remove unwanted, add wanted

kernelopts=$(grub2-editenv list | grep "^kernelopts" |
  sed 's/rhgb quiet//' )
        echo $kernelopts

grub2-editenv - set kernelopts="$kernelopts  quiet selinux=0 net.ifnames=0 audit=0"
        grub2-editenv list

The diagnostics looked OK, so I rebooted.
The reboot FAILED at switchroot.

Why did it fail,  and how can I undo the change?

The man page for grub-editenv says the grub environment is saved in
/boot/grub/grubenv .  Not quite;  it should say /boot/grub2/grubenv,
which is a symlink (on my UEFI system):
grubenv -> ../efi/EFI/fedora/grubenv
and that target file, normally, is missing.

By rebooting to my emergency F29 system I found that this file now
existed.  It contained the expected new kernel options, but was padded out
to a size of 1024 bytes with extraneous '#' characters.  Whether due to
my typing error or to some weird MSDOS artifact, I do not know.  I removed
them and rebooted - again the boot failed at the switchroot step.

Ultimately, I deleted (renamed) the grubenv file and then could boot
F30 again, but with the original default kernel options.

So F30 was released without ability to define alternate kernel options
sensibly.  And this was not a blocker???

Comment 12 Javier Martinez Canillas 2019-10-24 09:34:37 UTC
Since RHEL 8.1 the grubby tool updates the kernelopts variable in the grubenv file when using the --update-kernel=ALL option, instead of updating the individual BLS files.

Closing this bug as a duplicate of Bug #1690765 where that issue was fixed.

*** This bug has been marked as a duplicate of bug 1690765 ***


Note You need to log in before you can comment on or make changes to this bug.