Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 617535 - Upgrade when / is on ISCSI failed
Upgrade when / is on ISCSI failed
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: anaconda (Show other bugs)
6.0
All Linux
low Severity medium
: rc
: ---
Assigned To: Ales Kozumplik
Release Test Team
:
Depends On: 594411
Blocks:
  Show dependency treegraph
 
Reported: 2010-07-23 07:40 EDT by Alexander Todorov
Modified: 2014-09-30 19:39 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 594411
Environment:
Last Closed: 2010-12-06 11:10:39 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
grub.conf after 1st install (949 bytes, text/plain)
2010-07-23 08:24 EDT, Alexander Todorov
no flags Details
grub.conf after upgrade (1.92 KB, text/plain)
2010-07-23 08:25 EDT, Alexander Todorov
no flags Details
anaconda.log (13.90 KB, text/plain)
2010-07-23 08:25 EDT, Alexander Todorov
no flags Details
program.log (20.03 KB, text/plain)
2010-07-23 08:25 EDT, Alexander Todorov
no flags Details
storage.log (51.82 KB, text/plain)
2010-07-23 08:25 EDT, Alexander Todorov
no flags Details
syslog (37.96 KB, text/plain)
2010-07-23 08:26 EDT, Alexander Todorov
no flags Details
yum.log (2.79 KB, text/plain)
2010-07-23 08:26 EDT, Alexander Todorov
no flags Details

  None (edit)
Comment 1 Alexander Todorov 2010-07-23 07:41:50 EDT
Steps to reproduce: 

1) Install a system where /boot is on local disk and / on iSCSI
2) Upgrade this system to newer snapshot

Actual result: 
System doesn't boot. root device can't be found. 

Will try a simpler reproducer and upload logs.
Comment 2 Alexander Todorov 2010-07-23 08:24:41 EDT
Created attachment 433935 [details]
grub.conf after 1st install

The system has /boot and swap on local disk and / on iSCSI disk. No LVM.
Comment 3 Alexander Todorov 2010-07-23 08:25:14 EDT
Created attachment 433936 [details]
grub.conf after upgrade
Comment 4 Alexander Todorov 2010-07-23 08:25:27 EDT
Created attachment 433937 [details]
anaconda.log
Comment 5 Alexander Todorov 2010-07-23 08:25:44 EDT
Created attachment 433939 [details]
program.log
Comment 6 Alexander Todorov 2010-07-23 08:25:53 EDT
Created attachment 433940 [details]
storage.log
Comment 7 Alexander Todorov 2010-07-23 08:26:07 EDT
Created attachment 433941 [details]
syslog
Comment 8 Alexander Todorov 2010-07-23 08:26:18 EDT
Created attachment 433942 [details]
yum.log
Comment 9 Alexander Todorov 2010-07-23 08:27:13 EDT
(In reply to comment #3)
> Created an attachment (id=433936) [details]
> grub.conf after upgrade    

The interesting part here is that there's an entry for the new kernel as well as additional entry for the already installed kernel.
Comment 10 Hans de Goede 2010-07-23 08:35:23 EDT
Hi,

(In reply to comment #9)
> (In reply to comment #3)
> > Created an attachment (id=433936) [details] [details]
> > grub.conf after upgrade    
> 
> The interesting part here is that there's an entry for the new kernel as well
> as additional entry for the already installed kernel.    

Which is I must admit a bit weird, but other then that everything looks fine. Will the system still boot when you select the old kernel ?

Are you doing this inside a kvm ?

Can you see if there is an initramfs-.... file for the new kernel, and how its size compares to the one for the old kernel ?

If the system boots with the old kernel, can you rebuild the initramfs for the old kernel (first move the old one to .bak) and see if the old kernel still boots with a re-generated (so new) initramfs ?

Thanks,

Hans
Comment 11 Alexander Todorov 2010-07-26 08:39:56 EDT
(In reply to comment #10)
> Hi,
> 
> (In reply to comment #9)
> > (In reply to comment #3)
> > > Created an attachment (id=433936) [details] [details] [details]
> > > grub.conf after upgrade    
> > 
> > The interesting part here is that there's an entry for the new kernel as well
> > as additional entry for the already installed kernel.    
> 
> Which is I must admit a bit weird, but other then that everything looks fine.
> Will the system still boot when you select the old kernel ?
> 

As far as I remember booting the old kernel failed as well. 


> Are you doing this inside a kvm ?
> 

Yes.

> Can you see if there is an initramfs-.... file for the new kernel, and how its
> size compares to the one for the old kernel ?
> 
> If the system boots with the old kernel, can you rebuild the initramfs for the
> old kernel (first move the old one to .bak) and see if the old kernel still
> boots with a re-generated (so new) initramfs ?
> 

I will try to compare both initramfs images as soon as I get a system in Beaker. There seems to be a problem now (bug #618123).
Comment 12 Hans de Goede 2010-07-26 10:05:46 EDT
Hi,

(In reply to comment #11)
<snip>

> > Are you doing this inside a kvm ?
> > 
> 
> Yes.
>

Hmm, there is a kernel bug in RHEL-5 and RHEL-6 kernel hosts, which get exposed by doing installs inside kvm with recent RHEL-6 builds. This bug is causing a problem with memory corruption, which is causing all kind of random errors. I've a feeling that you are hitting this here.

Can you please try to reproduce using real hardware ?

Regards,

Hans
Comment 13 Alexander Todorov 2010-07-27 10:27:43 EDT
(In reply to comment #10)
> 
> Are you doing this inside a kvm ?
>

Next results are from bare metal. 
 
> Can you see if there is an initramfs-.... file for the new kernel, and how its
> size compares to the one for the old kernel ?
> 

After upgrade completed but while still in anaconda I've switched to tty2 and the results are:

# ls -lh initramfs-2.6.32-*
-rw-r--r--. 1 root root 19M 2010-07-27 10:20 initramfs-2.6.32-37.el6.x86_64.img
-rw-r--r--. 1 root root 19M 2010-07-27 10:21 initramfs-2.6.32-52.el6.x86_64.img

The system was able to boot and find the root fs after upgrade.


There are two issues here: 
1) grub.conf will add additional config entry for the old kernel
2) a KVM domU will not boot after upgrade while bare metal will
Comment 14 Hans de Goede 2010-07-27 11:20:14 EDT
Hi,

(In reply to comment #13)
> (In reply to comment #10)
> > 
> > Are you doing this inside a kvm ?
> >
> 
> Next results are from bare metal. 
> 
> > Can you see if there is an initramfs-.... file for the new kernel, and how its
> > size compares to the one for the old kernel ?
> > 
> 
> After upgrade completed but while still in anaconda I've switched to tty2 and
> the results are:
> 
> # ls -lh initramfs-2.6.32-*
> -rw-r--r--. 1 root root 19M 2010-07-27 10:20 initramfs-2.6.32-37.el6.x86_64.img
> -rw-r--r--. 1 root root 19M 2010-07-27 10:21 initramfs-2.6.32-52.el6.x86_64.img
> 
> The system was able to boot and find the root fs after upgrade.
> 

Ok, good!

> 
> There are two issues here: 
> 1) grub.conf will add additional config entry for the old kernel

Yes that is a (grubby) bug I'll gladly admit that. But that seems something to fix for 6.1 to me.

> 2) a KVM domU will not boot after upgrade while bare metal will    

That is very very likely bug 607650, which is a kernel issue on the *host* side (and yes Fedora hosts have the same problem). To verify you need to install a fixed kernel on the *host* side.

Regards,

Hans
Comment 15 Denise Dumas 2010-07-29 17:03:22 EDT
Please note that we explicitly do not support upgrades from one snapshot to another, or from RHEL5 to RHEL6. Only upgrades from one minor release to the next.
Comment 16 Ales Kozumplik 2010-10-08 09:40:01 EDT
> There are two issues here: 
> 1) grub.conf will add additional config entry for the old kernel
> 2) a KVM domU will not boot after upgrade while bare metal will

Hi Alex,

OK to close this one given that 594411 is in VERIFIED now and other problem was caused by (now fixed 607650)?

If in doubt can you please retest with the latest RHEL6 RC? (note that you need to have a fix on the host machine too if you want to test this in KVM).

Thanks.
Ales
Comment 17 Ales Kozumplik 2010-12-06 11:10:39 EST
Fixed already per comment 16.

Note You need to log in before you can comment on or make changes to this bug.