Bug 893390 - [Hyper-V]GRUB information lost when turn off guest with kernel updated from Hyper-V host
[Hyper-V]GRUB information lost when turn off guest with kernel updated from H...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: grubby (Show other bugs)
6.4
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Peter Jones
Release Test Team
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-01-09 05:05 EST by Shengnan Wang
Modified: 2013-04-01 08:23 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
When a Red Hat Enterprise Linux 6.4 guest updates the kernel and then the guest is turned of through Microsoft Hyper-V Manager, the guest fails to boot due to incomplete grub information. This is because the data is not synced properly to disk when the machine is turned off through Hyper-V Manager. To work around this problem, execute the "sync" command before turning the guest off.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-04-01 08:23:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
i386_348_After_Start.jpg (30.28 KB, image/jpeg)
2013-01-09 05:06 EST, Shengnan Wang
no flags Details
x86_64_GRUB_After_Start.jpg (61.85 KB, image/jpeg)
2013-01-09 05:07 EST, Shengnan Wang
no flags Details

  None (edit)
Description Shengnan Wang 2013-01-09 05:05:23 EST
Description of problem:
After a RHEL6.4 guest updated the kernel from -345 to -348 (rpm -ivh kernel-xxx) and then "Turn Off" through Hyper-V Manager, it failed
to boot up due to the incomplete grub information.

For i386 guest:
- when boot up with the new kernel (-348), it failed with error 'Error 2: Bad file or directory type'.(i386_348_After_Start.jpg)
- when boot up with the old kernel (-345), it failed with kernel panic because the grub information was not complete.

For x86_64 guest, there was no grub menu show up but run into grub command shell environment (x86_64_GRUB_After_Start.jpg). It could
be boot up successfully after manually specify the 'kernel' and 'initrd' in the grub command interface.

Version-Release number of selected component (if applicable):
Host: Windows 2008 R2 Hyper-V Server Core and Windows 2012 Hyper-V Server Core
Guest: Both i386 and x86_64 RHEL6.4 guest (2.6.32-348.el6)

How reproducible:
100%

Steps to Reproduce:
Pre-setup:
A RHEL6.4 guest was installed with kernel 2.6.32-345.el6. Let's take i686 for example (the steps for x86_64 were the same).

1.# rpm -ivh kernel-2.6.32-348.el6.i686.rpm
2.Turn off the guest via Hyper-V Manager. On Hyper-V manager, right click the guest in the Virtual Machines. Select "Turn Off...".
3.Start the guest via Hyper-V Manager
4.Go into the GRUB and start the guest with the new kernel (kernel-2.6.32-348.el6).
5.Go into the GRUB and start the guest with the previous kernel (kernel-2.6.32-345.el6)

Actual results:
For i386 guest,
At step4, the grub information was complete. However, failed to start the guest with error 'Error 2: Bad file or directory type'.(i386_348_After_Start.jpg)
At step5, kernel panic because the grub information was incomplete. The guest was started after correcting the grub. 
For x86_64 guest,
At step4 and step5, it run into grub command shell environment(x86_64_GRUB_After_Start.jpg).

Expected results:
For i386 guest,
At step4, started the guest successfully.
At step5, the information from the grub was complete. Started the guest successfully without call trace.
For x86_64 guest,
At step4 and step5, the grub menu shows up and the guest can boot up with both the new and the old kernels.

Additional info:
1) No this problem if run 'reboot' or 'shutdown -h 0' within the guest.
2) No this problem when down-grade the guest kernel.
3) If the guest has been reboot or shutdown within it, then there is no problem when "Turn Off" it via Hyper-V Manager.
Comment 1 Shengnan Wang 2013-01-09 05:06:33 EST
Created attachment 675377 [details]
i386_348_After_Start.jpg
Comment 2 Shengnan Wang 2013-01-09 05:07:11 EST
Created attachment 675378 [details]
x86_64_GRUB_After_Start.jpg
Comment 4 Shengnan Wang 2013-01-09 22:06:35 EST
Hi K.Y. ,

What do you think about this matter?

Thanks!
Comment 5 K. Y. Srinivasan 2013-01-10 10:42:18 EST
I have never seen something like this! Have you tested this on a bare maetal install to see if the problem can be reproduced there. I was under the impression that Hashir had tested all the upgrade paths. I am copying Hashir to get his input.
Comment 6 Václav Pavlín 2013-01-11 08:21:56 EST
Hi,

What i understand from your problem description is that the grub.conf was somehow corrupted:
 The guest was started after correcting the grub. 

If the problem is grub.conf, then it probably is not bug in grub itself, but rather in tool grubby.

So what did you mean by "correcting the grub"? Could you provide grub.conf causing this issue?
Comment 7 habdi 2013-01-11 10:35:14 EST
We tested  the upgrade paths from 6.3 to 6.4.

This is  unique way of testing kernel upgrades within 6.4 itself. 

Neither Hyper-V, nor LIS is responsible for updating the grub after a kernel upgrade... and I also don't believe that they are impeding this update in anyway since RHEL internal tools are successfully updating the grub upon reboot.  



Furthermore, it is good practice to Not Turn-off the system after a kernel upgrade like this  without rebooting but to use shutdown instead.

I would submit therefore that at best, this is a documentable best practices issue from our side 

Regards
Comment 8 Shengnan Wang 2013-01-14 06:24:01 EST
(In reply to comment #6)
> Hi,
> 
> What i understand from your problem description is that the grub.conf was
> somehow corrupted:
>  The guest was started after correcting the grub. 
> 

Yes.

> If the problem is grub.conf, then it probably is not bug in grub itself, but
> rather in tool grubby.
> 

The grub.conf file is right before 'turn off' the guest via Hyper-V Manager.
So i have no idea it is a bug about grub or grubby. 


> So what did you mean by "correcting the grub"? Could you provide grub.conf
> causing this issue?

For i386 guests:
When started the guest with the previous kernel, the grub was described as bellow:
'        root (hd0,0)
        kernel /vmlinuz-2.6.32-345.el6.i686 ro root=/dev/mapper/vg_dhcp6672126-lv_root rd_LVM_LV=vg_dh'
Modified the kernel line and added the initrd line to the grub:
'        root (hd0,0)
        kernel /vmlinuz-2.6.32-345.el6.i686 ro root=/dev/mapper/vg_dhcp6672126-lv_root rd_LVM_LV=vg_dhcp6672126/lv_swap rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_dhcp6672126/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
        initrd /initramfs-2.6.32-345.el6.i686.img'
Then started the guest with the previous kernel successfully. 

For x86_64 guest, the GRUB information lost. When modified the grub and added 'root','kernel','initrd', started the guest successfully. 


Met the same issue when test the upgrade to kernel-2.6.32-353.el6.
Comment 9 Václav Pavlín 2013-01-14 06:48:00 EST
Hi,

This really doesn't seem to be the grub issue. Grubby component is responsible for creating grub.conf, but I guess the problem will be in your Turning-Off machine by Hyper-V Manager - data is not synced properly to disk when you turn off the machine.

Try to use command "sync" before you turn off the machine, if it helps, I will close it as NOTABUG.

Habdi, maybe it would be useful to extend documentation of best practices as you noted.

Regards

Vaclav Pavlin
Comment 10 Deng Dewei 2013-01-18 01:04:49 EST
(In reply to comment #9)
Follow as steps described above, upgrade 345 -> 348, came up with same problem. 
> Hi,
> 
> This really doesn't seem to be the grub issue. Grubby component is
> responsible for creating grub.conf, but I guess the problem will be in your
> Turning-Off machine by Hyper-V Manager - data is not synced properly to disk
> when you turn off the machine.
> 
> Try to use command "sync" before you turn off the machine, if it helps, I
> will close it as NOTABUG.
> 

Before manually restart, execute #sync , have not found grub info missing, can always boot up successfully.

> Habdi, maybe it would be useful to extend documentation of best practices as
> you noted.
> 
> Regards
> 
> Vaclav Pavlin

Downgrade 348 -> 345. Do not run #sync before manually reboot.
As to x64, 40% grub.conf was somehow corrupted; 40% turned into grub shell interface, boot up after manually specify the 'kernel' and 'initrd' in the grub command interface failed, turn to panic. 
As to x86, 80% turn to be grub shell interface, boot up after manually specify the 'kernel' and 'initrd' in the grub command interface failed, turn to panic also.
When run #sync before manually restart, can boot up normally.

And I changed the component to Grubby.
Comment 11 RHEL Product and Program Management 2013-01-18 01:07:58 EST
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unable to address this
request at this time.

Red Hat invites you to ask your support representative to
propose this request, if appropriate, in the next release of
Red Hat Enterprise Linux.
Comment 12 habdi 2013-01-18 10:40:25 EST
I'll follow up internally and document this issue as a post update sync requirement before manual restart. Suggest we close this issue
Comment 13 Shengnan Wang 2013-01-21 03:43:52 EST
Hello Václav and Peter,
As there is no such problem found after executing command sync before Turn Off the guest, it's clearly to be a problem related to MSFT rather than grub/grubby. While, could this problem be closed as CANTFIX or WONTFIX? Also suggest to add a release note to this problem to inform the RHEL6.4 customer.
Comment 14 David Cantrell 2013-04-01 08:23:41 EDT
Closing per comment #13.  Leaving the technical notes text in the field so it remains in the release and/or technical notes.

Note You need to log in before you can comment on or make changes to this bug.