+++ This bug was initially created as a clone of Bug #1777874 +++
Description of problem:
The problem is with grub2-mkconfig, however in customer's scenario its being called via ansible playbook /usr/share/ansible/roles/tripleo-kernel/tasks/kernelargs.yml
===
Reproducer steps.
-> Install RHEL8.2 GA
-> Subscribe the server
-> Install tuned-profiles-cpu-partitioning-2.13.0-6.el8.noarch
# cat /boot/grub2/grubenv
# GRUB Environment Block
saved_entry=xxx-4.18.0-193.el8.x86_64
kernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet
boot_success=0
boot_indeterminate=0
-> Set isolated cpus in /etc/tuned/cpu-partitioning-variables.conf
# cat /etc/tuned/cpu-partitioning-variables.conf
# Examples:
# isolated_cores=2,4-7
# isolated_cores=2-23
#
# To disable the kernel load balancing in certain isolated CPUs:
# no_balance_cores=5-10
isolated_cores=1,2
# grep processor.: /proc/cpuinfo
processor : 0
processor : 1
processor : 2
processor : 3
-> Set tuned profile to 'cpu-partitioning'
# tuned-adm profile cpu-partitioning
# cat /boot/grub2/grubenv
# GRUB Environment Block
saved_entry=xxx-4.18.0-193.el8.x86_64
kernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet
boot_success=1
tuned_params=skew_tick=1 nohz=on nohz_full=1,2 rcu_nocbs=1,2 tuned.non_isolcpus=00000009 intel_pstate=disable nosoftlockup
tuned_initrd=(hd0,msdos1)/tuned-initrd.img
-> Simply run 'grub2-mkconfig' and notice the change in grubenv file.
# grub2-mkconfig
# cat /boot/grub2/grubenv
# GRUB Environment Block
saved_entry=xxx-4.18.0-193.el8.x86_64
kernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet $tuned_params
boot_success=1
tuned_params=skew_tick=1 nohz=on nohz_full=1,2 rcu_nocbs=1,2 tuned.non_isolcpus=00000009 intel_pstate=disable nosoftlockup
tuned_initrd=(hd0,msdos1)/tuned-initrd.img
-> on 'kernelopts' line we see '$tuned_params' got added which is why duplicate entries get populated
during boot.
# cat /proc/cmdline
BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-193.el8.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet skew_tick=1 nohz=on nohz_full=1,2 rcu_nocbs=1,2 tuned.non_isolcpus=00000009 intel_pstate=disable nosoftlockup skew_tick=1 nohz=on nohz_full=1,2 rcu_nocbs=1,2 tuned.non_isolcpus=00000009 intel_pstate=disable nosoftlockup
WORKAROUND:
==========
-> Ideally 'grub2-mkconfig' should not be used in RHEL8 since BLS is used
-> But if one want to still use 'grub2-mkconfig' they can use option '--no-grubenv-update'
which will not populate '$tuned_params' on 'kernelopts' line.
bz: https://bugzilla.redhat.com/show_bug.cgi?id=1777874
-> The problem is seen with any tuned profile which modifies kernel command line.
These include network-latency, cpu-partitioning, and realtime.
CUSTOMER's Environment
======================
-> In case of customer's scenario, I can see 'grub2-mkconfig' was ran by ansible. This is the reason post reboot
duplicate parameters got populated.
$ grep mkconfig var/log/messages-
Invoked with _raw_params=grub2-mkconfig -o /boot/grub2/grub.cfg warn=True _uses_shell=False stdin_add_newline=True strip_empty_ends=True argv=None chdir=None executable=None creates=None removes=None stdin=None
-> The ansible playbook file is /usr/share/ansible/roles/tripleo-kernel/tasks/kernelargs.yml
~~~~~~
43 # - name: Generate grub config
44 # command: "grub2-mkconfig -o {{ item.stat.path }}"
45 # when: item.stat.exists|bool
46 # loop: "{{ grub_stat.results }}"
~~~~~~
-> Customer is requesting an official fix.
I believe the ansible playbook is maintained by sbr-stack, hence requesting further investigation
from stack on 'grub2-mkconfig' command, whether any workaround with regards to ansible playbook
can be given to customer.
Customer claims that commenting out works but ansible process fails at various points.
NOTE
****
The problem here is caused by grub2-mkconfig, but its being called by ansible playbook 'kernelargs.yml'