Bug 1169344

+++ This bug was initially created as a clone of Bug #1085040 +++

On friday I started a rhel6 to rhel7 upgrade and let it run over the weekend. Apparently, this machine was previously a RHEL 5 machine before it was a rhel6 machine because before doing the upgrade it showed rhel5 grub artwork.   When I got into the office, the now rhel 7 machine was was sitting at a kernel panic.  It didn't recognize the filesystem root. After some troubleshooting, I discovered the problem was a missing initrd line in the grub (grub 1 !) config.

So it seems like there are 2 issues:

1) it's not upgrading to grub2
2) it's not adding the initrd to the grub1 config.

I don't know how much of this trouble is caused by preexisting wonkiness.  I do have a clone of the rhel6 VM from before the upgrade was started if that's useful.

I wasn't sure if I should file this against redhat-upgrade-tool or grubby, so feel free to reassign if necessary.

--- Additional comment from David Shea on 2014-04-07 11:42:20 EDT ---

Can you attach the grub.cfg from before and after the upgrade, as well as /var/log/redhat_upgrade_tool.log and /var/log/upgrade.log?

--- Additional comment from Ray Strode [halfline] on 2014-04-07 12:29:55 EDT ---

will attach those files. I actually wouldn't be surprised if this is caused by bug 1062714  .  the upgrade left me with broken python.

--- Additional comment from Ray Strode [halfline] on 2014-04-07 12:34:26 EDT ---



--- Additional comment from Ray Strode [halfline] on 2014-04-07 12:34:49 EDT ---



--- Additional comment from Ray Strode [halfline] on 2014-04-07 12:35:15 EDT ---



--- Additional comment from Ray Strode [halfline] on 2014-04-07 12:35:42 EDT ---



--- Additional comment from David Shea on 2014-04-07 13:39:58 EDT ---

This is similar to bug 1062714, but yum appears to be making some decisions that don't make any sense. The logs indicate grubby-7.0.15-5.el6.x86_64 is being upgraded to grubby-8.28-8.el7.x86_64, but then there's:

[   151.433] (DD) redhat_upgrade_tool.depsolve:format_missing_requires() MISSING REQ: kernel-2.6.32-358.14.1.el6.x86_64 requires /sbin/new-kernel-pkg
[   151.436] (DD) redhat_upgrade_tool.yum:_requiringFromTransaction() TSINFO: Marking grubby-7.0.15-5.el6.i686 as install for kernel-2.6.32-358.14.1.el6.x86_64

The kernel package is being removed, so I assume it's for the preun dependency, but the dependencies in the rpm are for  grubby >= 7.0.4-1 and /sbin/new-kernel-pkg, both of which are provided by grubby-8.28-8. Unless this is related to UsrMove, I guess, but the RHEL 7 kernel.spec still uses /sbin and not /usr/sbin as the dependency.

Reassigning to yum.

--- Additional comment from RHEL Product and Program Management on 2014-04-07 13:43:49 EDT ---

This request has been proposed as a blocker, but a release flag has
not been requested. Please set a release flag to ? to ensure we may
track this bug against the appropriate upcoming release, and reset
the blocker flag to ?.

--- Additional comment from RHEL Product and Program Management on 2014-04-07 13:52:17 EDT ---

Since this bug report was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from James Antill on 2014-06-16 09:53:51 EDT ---

> The kernel package is being removed, so I assume it's for the preun
> dependency, but the dependencies in the rpm are for  grubby >= 7.0.4-1 and
> /sbin/new-kernel-pkg, both of which are provided by grubby-8.28-8. Unless
> this is related to UsrMove, I guess, but the RHEL 7 kernel.spec still uses
> /sbin and not /usr/sbin as the dependency.

 8.28 doesn't provide that.

% rpm -qlp 7.0.15/5.el6/i686/grubby-7.0.15-5.el6.i686.rpm
/sbin/grubby
/sbin/installkernel
/sbin/new-kernel-pkg
/usr/share/doc/grubby-7.0.15
/usr/share/doc/grubby-7.0.15/COPYING
/usr/share/man/man8/grubby.8.gz
/usr/share/man/man8/installkernel.8.gz
/usr/share/man/man8/new-kernel-pkg.8.gz
% rpm -qlp 8.28/8.el7/i686/grubby-8.28-8.el7.i686.rpm
/usr/sbin/grubby
/usr/sbin/installkernel
/usr/sbin/new-kernel-pkg
/usr/share/doc/grubby-8.28
/usr/share/doc/grubby-8.28/COPYING
/usr/share/man/man8/grubby.8.gz
/usr/share/man/man8/installkernel.8.gz
/usr/share/man/man8/new-kernel-pkg.8.gz
% rpm -qp --provides 8.28/8.el7/i686/grubby-8.28-8.el7.i686.rpm
grubby = 8.28-8.el7
grubby(x86-32) = 8.28-8.el7


> 
> Reassigning to yum.

--- Additional comment from Ray Strode [halfline] on 2014-06-19 13:55:16 EDT ---

James, did you mean to close this as NOTABUG versus move it to grubby?

to be clear the bug (is the very serious iss) "RHEL6 to RHEL7 upgrade doesn't work".  Do you believe that problem to be gone?

Are you were you just looking at this report through yum goggles ?

--- Additional comment from Petr Hracek on 2014-10-16 09:00:25 EDT ---

During the upgrade RHEL6.6->RHEL7.1 the grub.conf loses the initrd again.
Reopen the bug.

It causes kernel panic.

--- Additional comment from  on 2014-10-16 11:44:30 EDT ---

It seems as problem inside grubby. From audit.log (I will add it here soon) only grubby modify grub.conf (if I understand this output well)
In upgrade.log you can see attempt for opening grub.cfg (I don't know if this is useful info, when we don't work with grub2 here).

--- Additional comment from  on 2014-10-16 11:46:53 EDT ---



--- Additional comment from  on 2014-10-17 03:00:18 EDT ---

Similar problem is now actual on Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=725185

In addition: I can't run grubby anymore after migration due to segfault error. But I don't investigate it deeper. May it's caused by mix of grub/grub2.

--- Additional comment from Petr Hracek on 2014-11-07 03:43:49 EST ---

Hello Peter,

is it possible to fix the problem soon?

This bug blocks upgrades from RHEL6.6->RHEL7.1.
Do you need more information from us like logs?

--- Additional comment from Petr Hracek on 2014-11-07 07:44:37 EST ---

I think that the bug is not related to redhat-upgrade-tool.
Reassigning to grubby.

As was mentioned above during upgrade initrd row is missing in grub.conf

--- Additional comment from RHEL Product and Program Management on 2014-11-07 07:48:00 EST ---

Since this bug report was entered in bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

--- Additional comment from Ray Strode [halfline] on 2014-11-07 15:05:16 EST ---

Petr, assuming it's the same issue as before, iirc, the issue was python wasn't upgraded properly.  So it wouldn't be a grubby issue, since clearly grubby can't be expected to work with a broken python installation on the system.

--- Additional comment from Juraj Marko on 2014-11-26 09:25:40 EST ---

I'm also able to reproduce this bug, and from my point of view it is a blocker and also a test blocker, since all our automation is not working because of this bug. I'm not sure which component is causing this issue, but from a qa point of view it is also a regression since on upgrade from rhel6.6 to 7.0 it was working OK.

--- Additional comment from  on 2014-11-27 03:42:37 EST ---

In my point of view it's not caused by python - after my few tests python is working as well as usualy. I don't see any problem here. You can try it yourself if you find unwanted behaviour of python on your machine after migration. IMHO that's really problem of grubby. We can create content with postscript as ugly workround, but really this should be fixed properly and not in this way of workrounds. Again, what do you think Peter, about it? My apologize that I prefer your solution/opinion before my (or anyone else) drilling in grubby.

--- Additional comment from  on 2014-11-27 04:13:31 EST ---

P.S. workround for manual testing:
boot to rescue mode, add initrd <your ramfs image> to you /boot/grub/grub.conf, may still:
 initrd /initramfs-3.10.0-175.el7.x86_64.img

save, reboot, wait for modificationt contexts for SELinux, after reboot should be everything OK.

--- Additional comment from Petr Hracek on 2014-11-27 05:37:19 EST ---

It seems that libsasl2.so.2 is required but libsasl2.so.3.0.0 is on the system

--- Additional comment from Petr Hracek on 2014-11-27 05:38:02 EST ---

But problem with missing with initrd in /boot/grub/grub.conf is still there.

--- Additional comment from  on 2014-11-27 06:22:57 EST ---

Yes, it seems that this could be source of problems. grubby is indirectly dependent on cyrus-sasl-lib, which contains libsasl2.so*
(see for rhel-7: repoquery -R --resolve --recursive grubby | grep -i cyrus).

Change component to cyrus-sasl-lib.

--- Additional comment from Marian Ganisin on 2014-11-28 07:50:43 EST ---

Based on my analysis for issue occuring in 7.1 seems to be a regression in grubby.

This line from postrans of kernel package:

/usr/sbin/new-kernel-pkg --package kernel --mkinitrd --dracut --depmod --update 3.10.0-201.el7.x86_64 || exit $?

initiates different grubby commands (those updating grub.cfg) with packages from 7.0 and 7.1

With grubby-8.28-11.el7.x86_64 (version from RHEL-7.1) new-kernel-pkg mentioned above uses this to update the grub.cfg:

/sbin/grubby --grub -c /boot/grub/grub.conf --update-kernel=/boot/vmlinuz-3.10.0-201.el7.x86_64 --initrd /boot/initramfs-3.10.0-201.el7.x86_64.img --args=" LANG=en_US.UTF-8" --title="Red Hat Enterprise Linux Server (3.10.0-201.el7.x86_64) 7.1 (Maipo)$debugtitle"

While with grubby-8.28-8.el7.x86_64 (version from RHEL-7.0) command is:

/sbin/grubby --grub -c /boot/grub/grub.conf --update-kernel=/boot/vmlinuz-3.10.0-201.el7.x86_64 --initrd /boot/initramfs-3.10.0-201.el7.x86_64.img --args=" LANG=en_US.UTF-8"

The only difference is in --title option. I manually verified on RHEL-7.1 got by Inplace upgrade that --title does the magic. When defined the grubby does not update grub.cfg, while without --title it does the thing.

It's good idea to repeat that described issue happens after RHEL-7 Inplace upgrade when RHEL-6.6 is updated to RHEL-7.1 which always keeps _grub_  in use also on RHEL-7 (not the grub2 as usual), thus this is all about configuring grub-0.97 on RHEL-7.

Finally as Juraj Marko mentioned, this is blocker and TestBlocker for RHEL7 Inplace Upgrades feature as the system can not boot after inplace upgrade unless grub configuration is manually updated.

--- Additional comment from Frantisek Kluknavsky on 2014-11-28 15:01:13 EST ---

A temporary workaround: http://git.app.eng.bos.redhat.com/git/preupgrade-assistant-contents-users.git/tree/RHEL6_7/system/grubby