Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1441797 - cpu-partitioning: new kernels don't get $tuned_initrd
cpu-partitioning: new kernels don't get $tuned_initrd
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned (Show other bugs)
7.4
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Jaroslav Škarvada
Tereza Cerna
:
Depends On:
Blocks: 1394932
  Show dependency treegraph
 
Reported: 2017-04-12 14:13 EDT by Luiz Capitulino
Modified: 2017-08-01 08:35 EDT (History)
4 users (show)

See Also:
Fixed In Version: tuned-2.8.0-2.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-01 08:35:21 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2102 normal SHIPPED_LIVE tuned bug fix and enhancement update 2017-08-01 12:07:33 EDT

  None (edit)
Description Luiz Capitulino 2017-04-12 14:13:35 EDT
Description of problem:

When installing a new kernel on a system where the cpu-partitioning profile is applied, the new kernel won't get the $tuned_initrd parameter in grub.cfg. Which in turn means early boot-time zero-loss configuration won't be applied. Which in turn means no zero-loss.


Version-Release number of selected component (if applicable): tuned-2.8.0-1.el7	


How reproducible:


Steps to Reproduce:
1. Install, setup and active cpu-partitioning profile
2. Install a new kernel
3. Check grub.cfg for the initrd16 line for the new kernel, it won't have $tuned_initrd
Comment 2 Jaroslav Škarvada 2017-04-12 14:15:58 EDT
It's not an regression, it worked always this way, but there is a room for improvement.
Comment 5 Jaroslav Škarvada 2017-04-13 10:08:26 EDT
It fully relies on the kernel %post scripts, which utilizes new-kernel-pkg helper which is proved by grubby package.

The new-kernel-pkg copies the kernel command line from the default kernel, but it rewrites the initrd settings and it doesn't count with custom initrd overlays. I think this should be addressed as a grubby bug.

E.g.:
the old kernel in grub.cfg:
initrd16 /initramfs-3.10.0-349.el7.x86_64.img $tuned_initrd

When new kernel is installed new-kernel-pkg adds the following to grub.cfg:
initrd16 /initramfs-3.10.0-647.el7.x86_64.img

But it didn't copy the initrd overlays, i.e. $tuned_initrd in this case.

I am afraid we cannot do much about it in Tuned, the only way is to setup fanotify watch on the grub.cfg file and when it's changed then re-patch it. But it would be really dirty hack that could have various bad consequences, races could happen. Tuned is patching the grub.cfg safe way, and replace it in one atomic operation, but other tools may not do it this way. Such tool (e.g. in %post script) could start patching the grub.cfg in-place, Tuned would spot it and patch partially written file, then the tool would add something, Tuned would repatch it and get the config to bad state. Bad things could happen. This is not the way to go.

This should be fixed the right way on the right place (grubby) or we should document that it requires workarounds. 

The only thing we can probably do in Tuned is to improve the grub.cfg patching process. I.e. when the profile is applied or Tuned started, add the initrd overlays to new kernels.

Then workarounds would be:
- upon kernel upgrade restart tuned (this can be documented)
- or just reboot, the grub.cfg will be patched on boot and the settings will be applied on another boot

So dev ack from me for improvement of the patching process for above workarounds to work, dev nak design for different Tuned workarounds.
Comment 6 Luiz Capitulino 2017-04-13 10:22:45 EDT
Just to confirm I understood: grubby is re-writting the initrd16 line but for the linux16 it does the right thing (as this line does get $tuned_params for new kernels)?

In that case I agree with you this is a grubby issue, as it should be treating linux16 and initrd16 the same way in that regard.

The only question is whether this is a blocker bug for it the same way it is for tuned.
Comment 7 Jaroslav Škarvada 2017-04-13 10:32:13 EDT
(In reply to Luiz Capitulino from comment #6)
> Just to confirm I understood: grubby is re-writting the initrd16 line but
> for the linux16 it does the right thing (as this line does get $tuned_params
> for new kernels)?
Yes exactly, it copies the linux16 content from the default kernel, for initrd16 the content is not copied.

> 
> In that case I agree with you this is a grubby issue, as it should be
> treating linux16 and initrd16 the same way in that regard.
> 
I think the idea behind is that the kernel boot parameters aren't kernel specific, i.e. they do not change with kernel update, but the dracut initrd images changes. Of course this is not true for custom initrd overlays which don't require re-generation.

> The only question is whether this is a blocker bug for it the same way it is
> for tuned.

I think it worked always this way, probably nobody complained as custom initrd overlays are probably not very used. But I think it should be improved.
Comment 8 Luiz Capitulino 2017-04-13 10:43:02 EDT
Would move this bug to grubby then as you did the debugging?
Comment 9 Jaroslav Škarvada 2017-04-13 10:48:32 EDT
(In reply to Luiz Capitulino from comment #8)
> Would move this bug to grubby then as you did the debugging?

I will probably file new bug for the report to be more clear.
Comment 10 Jaroslav Škarvada 2017-04-13 10:49:21 EDT
Also there is another workaround which should work now out of the box. It is to run grub2-mkconfig after kernel update, which is the grub2 upstream intended way how to upgrade grub.cfg (grubby approach for patching is more like hack), e.g. on non EFI RHEL-7:
# grub2-mkconfig -o /boot/grub2/grub.cfg

On EFI the path is different.

This should generate the correct grub.cfg, because Tuned installs the mkconfig template.
Comment 11 Jaroslav Škarvada 2017-04-13 10:55:24 EDT
(In reply to Jaroslav Škarvada from comment #9)
> (In reply to Luiz Capitulino from comment #8)
> > Would move this bug to grubby then as you did the debugging?
> 
> I will probably file new bug for the report to be more clear.

Bug 1442117.
Comment 12 Jaroslav Škarvada 2017-04-13 18:00:53 EDT
(In reply to Jaroslav Škarvada from comment #10)
> Also there is another workaround which should work now out of the box. It is
> to run grub2-mkconfig after kernel update, which is the grub2 upstream
> intended way how to upgrade grub.cfg (grubby approach for patching is more
> like hack), e.g. on non EFI RHEL-7:
> # grub2-mkconfig -o /boot/grub2/grub.cfg
> 
> On EFI the path is different.
> 
> This should generate the correct grub.cfg, because Tuned installs the
> mkconfig template.

This specific workaround is blocked by bug 1427899.
Comment 13 Jaroslav Škarvada 2017-04-13 18:08:39 EDT
(In reply to Jaroslav Škarvada from comment #5)
> Then workarounds would be:
> - upon kernel upgrade restart tuned (this can be documented)
> - or just reboot, the grub.cfg will be patched on boot and the settings will
> be applied on another boot

These workarounds should work after the following upstream commit:
https://github.com/redhat-performance/tuned/commit/f180a80013e18a5e64b8a200dc7823ac232622f1
Comment 15 Luiz Capitulino 2017-04-18 11:33:02 EDT
I verified that the restart workaround works. That is, restarting tuned after a kernel installation causes $tuned_initrd to be added to the initrd16 line.

This is a good workaround. My only concern is that people scripting the following:

1. Install kernel
2. Restart tuned
3. Reboot

Will likely get an unbootable system because of bug 1443142.
Comment 16 Tereza Cerna 2017-04-20 08:13:14 EDT
@Luiz: Thank you for your testing.

Switching to VERIFIED (SanityOnly) status based on successful test by lcapitulino.
Comment 17 errata-xmlrpc 2017-08-01 08:35:21 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2102

Note You need to log in before you can comment on or make changes to this bug.