RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1441797 - cpu-partitioning: new kernels don't get $tuned_initrd
Summary: cpu-partitioning: new kernels don't get $tuned_initrd
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned
Version: 7.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Jaroslav Škarvada
QA Contact: Tereza Cerna
URL:
Whiteboard:
Depends On:
Blocks: 1394932
TreeView+ depends on / blocked
 
Reported: 2017-04-12 18:13 UTC by Luiz Capitulino
Modified: 2017-08-01 12:35 UTC (History)
4 users (show)

Fixed In Version: tuned-2.8.0-2.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 12:35:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2102 0 normal SHIPPED_LIVE tuned bug fix and enhancement update 2017-08-01 16:07:33 UTC

Description Luiz Capitulino 2017-04-12 18:13:35 UTC
Description of problem:

When installing a new kernel on a system where the cpu-partitioning profile is applied, the new kernel won't get the $tuned_initrd parameter in grub.cfg. Which in turn means early boot-time zero-loss configuration won't be applied. Which in turn means no zero-loss.


Version-Release number of selected component (if applicable): tuned-2.8.0-1.el7	


How reproducible:


Steps to Reproduce:
1. Install, setup and active cpu-partitioning profile
2. Install a new kernel
3. Check grub.cfg for the initrd16 line for the new kernel, it won't have $tuned_initrd

Comment 2 Jaroslav Škarvada 2017-04-12 18:15:58 UTC
It's not an regression, it worked always this way, but there is a room for improvement.

Comment 5 Jaroslav Škarvada 2017-04-13 14:08:26 UTC
It fully relies on the kernel %post scripts, which utilizes new-kernel-pkg helper which is proved by grubby package.

The new-kernel-pkg copies the kernel command line from the default kernel, but it rewrites the initrd settings and it doesn't count with custom initrd overlays. I think this should be addressed as a grubby bug.

E.g.:
the old kernel in grub.cfg:
initrd16 /initramfs-3.10.0-349.el7.x86_64.img $tuned_initrd

When new kernel is installed new-kernel-pkg adds the following to grub.cfg:
initrd16 /initramfs-3.10.0-647.el7.x86_64.img

But it didn't copy the initrd overlays, i.e. $tuned_initrd in this case.

I am afraid we cannot do much about it in Tuned, the only way is to setup fanotify watch on the grub.cfg file and when it's changed then re-patch it. But it would be really dirty hack that could have various bad consequences, races could happen. Tuned is patching the grub.cfg safe way, and replace it in one atomic operation, but other tools may not do it this way. Such tool (e.g. in %post script) could start patching the grub.cfg in-place, Tuned would spot it and patch partially written file, then the tool would add something, Tuned would repatch it and get the config to bad state. Bad things could happen. This is not the way to go.

This should be fixed the right way on the right place (grubby) or we should document that it requires workarounds. 

The only thing we can probably do in Tuned is to improve the grub.cfg patching process. I.e. when the profile is applied or Tuned started, add the initrd overlays to new kernels.

Then workarounds would be:
- upon kernel upgrade restart tuned (this can be documented)
- or just reboot, the grub.cfg will be patched on boot and the settings will be applied on another boot

So dev ack from me for improvement of the patching process for above workarounds to work, dev nak design for different Tuned workarounds.

Comment 6 Luiz Capitulino 2017-04-13 14:22:45 UTC
Just to confirm I understood: grubby is re-writting the initrd16 line but for the linux16 it does the right thing (as this line does get $tuned_params for new kernels)?

In that case I agree with you this is a grubby issue, as it should be treating linux16 and initrd16 the same way in that regard.

The only question is whether this is a blocker bug for it the same way it is for tuned.

Comment 7 Jaroslav Škarvada 2017-04-13 14:32:13 UTC
(In reply to Luiz Capitulino from comment #6)
> Just to confirm I understood: grubby is re-writting the initrd16 line but
> for the linux16 it does the right thing (as this line does get $tuned_params
> for new kernels)?
Yes exactly, it copies the linux16 content from the default kernel, for initrd16 the content is not copied.

> 
> In that case I agree with you this is a grubby issue, as it should be
> treating linux16 and initrd16 the same way in that regard.
> 
I think the idea behind is that the kernel boot parameters aren't kernel specific, i.e. they do not change with kernel update, but the dracut initrd images changes. Of course this is not true for custom initrd overlays which don't require re-generation.

> The only question is whether this is a blocker bug for it the same way it is
> for tuned.

I think it worked always this way, probably nobody complained as custom initrd overlays are probably not very used. But I think it should be improved.

Comment 8 Luiz Capitulino 2017-04-13 14:43:02 UTC
Would move this bug to grubby then as you did the debugging?

Comment 9 Jaroslav Škarvada 2017-04-13 14:48:32 UTC
(In reply to Luiz Capitulino from comment #8)
> Would move this bug to grubby then as you did the debugging?

I will probably file new bug for the report to be more clear.

Comment 10 Jaroslav Škarvada 2017-04-13 14:49:21 UTC
Also there is another workaround which should work now out of the box. It is to run grub2-mkconfig after kernel update, which is the grub2 upstream intended way how to upgrade grub.cfg (grubby approach for patching is more like hack), e.g. on non EFI RHEL-7:
# grub2-mkconfig -o /boot/grub2/grub.cfg

On EFI the path is different.

This should generate the correct grub.cfg, because Tuned installs the mkconfig template.

Comment 11 Jaroslav Škarvada 2017-04-13 14:55:24 UTC
(In reply to Jaroslav Škarvada from comment #9)
> (In reply to Luiz Capitulino from comment #8)
> > Would move this bug to grubby then as you did the debugging?
> 
> I will probably file new bug for the report to be more clear.

Bug 1442117.

Comment 12 Jaroslav Škarvada 2017-04-13 22:00:53 UTC
(In reply to Jaroslav Škarvada from comment #10)
> Also there is another workaround which should work now out of the box. It is
> to run grub2-mkconfig after kernel update, which is the grub2 upstream
> intended way how to upgrade grub.cfg (grubby approach for patching is more
> like hack), e.g. on non EFI RHEL-7:
> # grub2-mkconfig -o /boot/grub2/grub.cfg
> 
> On EFI the path is different.
> 
> This should generate the correct grub.cfg, because Tuned installs the
> mkconfig template.

This specific workaround is blocked by bug 1427899.

Comment 13 Jaroslav Škarvada 2017-04-13 22:08:39 UTC
(In reply to Jaroslav Škarvada from comment #5)
> Then workarounds would be:
> - upon kernel upgrade restart tuned (this can be documented)
> - or just reboot, the grub.cfg will be patched on boot and the settings will
> be applied on another boot

These workarounds should work after the following upstream commit:
https://github.com/redhat-performance/tuned/commit/f180a80013e18a5e64b8a200dc7823ac232622f1

Comment 15 Luiz Capitulino 2017-04-18 15:33:02 UTC
I verified that the restart workaround works. That is, restarting tuned after a kernel installation causes $tuned_initrd to be added to the initrd16 line.

This is a good workaround. My only concern is that people scripting the following:

1. Install kernel
2. Restart tuned
3. Reboot

Will likely get an unbootable system because of bug 1443142.

Comment 16 Tereza Cerna 2017-04-20 12:13:14 UTC
@Luiz: Thank you for your testing.

Switching to VERIFIED (SanityOnly) status based on successful test by lcapitulino.

Comment 17 errata-xmlrpc 2017-08-01 12:35:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2102


Note You need to log in before you can comment on or make changes to this bug.