Bug 561221 - Snapshot of guest suffers kernel panic when installed with virtio block and boot with ide block
Snapshot of guest suffers kernel panic when installed with virtio block and b...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kvm (Show other bugs)
5.5
All Linux
high Severity high
: rc
: ---
Assigned To: Kevin Wolf
Virtualization Bugs
:
Depends On:
Blocks: Rhel5KvmTier3
  Show dependency treegraph
 
Reported: 2010-02-02 22:45 EST by Qunfang Zhang
Modified: 2013-01-09 17:16 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-01-14 09:04:08 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
rhel5.4 guest kernel panic (23.60 KB, image/png)
2010-02-02 22:45 EST, Qunfang Zhang
no flags Details
windows guest kernel panic (326.62 KB, image/png)
2010-02-02 22:45 EST, Qunfang Zhang
no flags Details

  None (edit)
Description Qunfang Zhang 2010-02-02 22:45:11 EST
Created attachment 388439 [details]
rhel5.4 guest kernel panic

Description of problem:
I install a rhel5.4 guest with virtio block, and create snapshot chain by:
#qemu-img create -f qcow2 -F raw -b rhel5.4-64.raw QCOW2_SPARSE_A
#qemu-img create -f qcow2 -F  qcow2 -b QCOW2_SPARSE_A QCOW2_SPARSE_B

Then boot the snapshot B with virtio block first,it works well.So I shutdown it and then boot it with ide block this time. Meet a kernel panic.
If i boot the snapshot B with ide block first, then shutdown it and boot again with virtio block, it will works well.
This issue also exists on windows guest.

Version-Release number of selected component (if applicable):
kernel-2.6.18-186.el5
kvm-83-155.el5

How reproducible:
100%

Steps to Reproduce:
1.Install a rhel5.4 guest with virtio block with the following command:
/usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -smp 2 -m 2G -drive file=/media/boot.iso,media=cdrom,if=ide -drive file=/home/qcow2/rhel5.4-64.raw,media=disk,if=virtio,boot=on -net nic,vlan=0,macaddr=10:20:1a:4a:40:5f,model=virtio -net tap,vlan=0,script=/etc/qemu-ifup -cpu qemu64,+sse2 -uuid `uuidgen` -vnc :10 -monitor stdio -boot d

2.After installation,shutdown the guest and create snapshot chain:
#qemu-img create -f qcow2 -F raw -b rhel5.4-64.raw QCOW2_SPARSE_A
#qemu-img create -f qcow2 -F  qcow2 -b QCOW2_SPARSE_A QCOW2_SPARSE_B

3.boot snapshot B with virtio block first
/usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -smp 2 -m 2G -drive file=/home/qcow2/QCOW2_SPARSE_B,media=disk,if=virtio,boot=on -net nic,vlan=0,macaddr=10:20:1a:4a:40:5f,model=virtio -net tap,vlan=0,script=/etc/qemu-ifup -cpu qemu64,+sse2 -uuid `uuidgen` -vnc :10 -monitor stdio -boot c

4.shutdown the snapshot B and boot it again with ide block.
/usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -smp 2 -m 2G -drive file=/home/qcow2/QCOW2_SPARSE_B,media=disk,if=ide -net nic,vlan=0,macaddr=10:20:1a:4a:40:5f,model=virtio -net tap,vlan=0,script=/etc/qemu-ifup -cpu qemu64,+sse2 -uuid `uuidgen` -vnc :10 -monitor stdio -boot c
  
Actual results:
Guest kernel panic (attachment will be added.)

Expected results:
Guest can boot with ide block.

Additional info:
host info: 
processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Quad CPU    Q9550  @ 2.83GHz
stepping	: 10
cpu MHz		: 2826.232
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips	: 5652.33
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 1 Qunfang Zhang 2010-02-02 22:45:45 EST
Created attachment 388440 [details]
windows guest kernel panic
Comment 2 Qunfang Zhang 2010-02-03 00:13:54 EST
Re-test the issue and boot the snapshot B with "-serial stdio" in command line.
Paste the log grabbed from guest:


 Booting 'Red Hat Enterprise Linux Server (2.6.18-164.el5)'

root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet cons
ole=ttyS0
   [Linux-bzImage, setup=0x1e00, size=0x1d5c1c]
initrd /initrd-2.6.18-164.el5.img
   [Linux-initrd @ 0x37cbd000, 0x332566 bytes]

Memory for crash kernel (0x0 to 0x0) notwithin permissible range
PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
�Red Hat nash version 5.1.19.6 starting
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
  2 logical volume(s) in volume group "VolGroup00" now active
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667905: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667963: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 196251: bad block 197645
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667961: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667907: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667969: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667970: bad block 1672200
/etc/sysconfig/i18n: line 1: syntax error near unexpected token `newline'
/etc/sysconfig/i18n: line 1: `             </entry>'
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668573: bad block 1672200
		Welcome to EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668112: bad block 1672200
Red Hat Enterprise Linux Server
		Press 'I' to enter interactive startup.
Setting clock  (utc): Wed Feb  3 13:08:57 CST 2010 [  OK  ]
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668304: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668526: bad block 1672200
/etc/sysconfig/i18n: line 1: syntax error near unexpected token `newline'
/etc/sysconfig/i18n: line 1: `             </entry>'
Starting udev: EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668519: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668520: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667935: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668386: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668387: bad block 1672200
error parsing "/etc/makedev.d/01alsa": bad major in "c 600 root root        [defaults]"
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668305: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668521: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668308: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668522: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668523: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668618: bad block 1672200
udevd[471]: add_to_rules: invalid rule '/etc/udev/rules.d/60-net.rules:1'
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668531: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668911: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668912: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1669183: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668356: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668524: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668525: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668365: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668496: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668515: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1667914: bad block 1672193
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668909: bad block 1672200
[  OK  ]
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668486: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668517: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668518: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668617: bad block 1672200
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668572: bad block 1672200
Setting hostname localhost.localdomain:  [  OK  ]
DM multipath kernel driver not loaded
Setting up Logical Volume Management:   2 logical volume(s) in volume group "VolGroup00" now active
[  OK  ]
EXT3-fs error (device dm-0): ext3_xattr_block_get: inode 1668616: bad block 1672200
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/VolGroup00/LogVol00 
/dev/VolGroup00/LogVol00 contains a file system with errors, check forced.
/dev/VolGroup00/LogVol00: Inode 7, i_blocks is 56088, should be 55800.  FIXED.
/dev/VolGroup00/LogVol00: Inode 196260 has a bad extended attribute block 197646.  

/dev/VolGroup00/LogVol00: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
	(i.e., without -a or -p options)
[FAILED]

*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
*** Warning -- SELinux is active
*** Disabling security enforcement for system recovery.
*** Run 'setenforce 1' to reenable.
Give root password for maintenance
(or type Control-D to continue):
Comment 3 Dor Laor 2010-02-09 04:48:22 EST
Does it work without snapshots? On windows for sure you cannot change the block device without changing the guest configuration. On Linux I'm not sure, it might be so too.
Comment 4 Qunfang Zhang 2010-02-09 05:00:57 EST
(In reply to comment #3)
> Does it work without snapshots? 
Without snapshot, guest will work well when change block between virtio and ide.

> On windows for sure you cannot change the block
> device without changing the guest configuration. On Linux I'm not sure, it
> might be so too.    
I install the guest with virtio block, then create snapshot A and B.

                         guest     snapshot A    snapshot B
1st boot with virtio       OK          OK          OK  
2nd boot with ide          OK          OK        *failed*
---------------------------------------------------------------
with different sequence, first ide then virtio:

                         guest     snapshot A    snapshot B
1st boot with IDE          OK          OK          OK  
2nd boot with virtio       OK          OK          OK
Comment 5 Kevin Wolf 2010-02-09 05:04:17 EST
In my experience switching between IDE and virtio works just fine on Linux, so I'd expect this to work. This needs some more detailed investigation.

The Windows case, on the other hand, doesn't look like a bug to me.
Comment 6 Kevin Wolf 2010-02-09 05:06:48 EST
(In reply to comment #4)
>                          guest     snapshot A    snapshot B
> 1st boot with virtio       OK          OK          OK  
> 2nd boot with ide          OK          OK        *failed*

What happens with a second virtio boot?

> with different sequence, first ide then virtio:
> 
>                          guest     snapshot A    snapshot B
> 1st boot with IDE          OK          OK          OK  
> 2nd boot with virtio       OK          OK          OK    

And here with both IDE and virtio in a third boot?
Comment 7 Qunfang Zhang 2010-02-09 23:27:16 EST
(In reply to comment #6)
> (In reply to comment #4)
> >                          guest     snapshot A    snapshot B
> > 1st boot with virtio       OK          OK          OK  
> > 2nd boot with ide          OK          OK        *failed*
> 
> What happens with a second virtio boot?

All work well with a second virtio boot.

> 
> > with different sequence, first ide then virtio:
> > 
> >                          guest     snapshot A    snapshot B
> > 1st boot with IDE          OK          OK          OK  
> > 2nd boot with virtio       OK          OK          OK    
> 
> And here with both IDE and virtio in a third boot?    

All work well with both IDE  and virtio in a third boot.

That is to say, this issue only happens when first boot with virtio then with ide on the snapshotB.
Comment 8 Dor Laor 2010-05-20 08:43:26 EDT
would it be helpful to test it over rhel6?
Comment 9 Qunfang Zhang 2010-05-20 22:57:45 EDT
(In reply to comment #8)
> would it be helpful to test it over rhel6?    

Test on RHEL6 host with the same steps in bug description, and this issue does not exist.

qemu-kvm version: qemu-kvm-0.12.1.2-2.54.el6.x86_64

kernel: 2.6.32-25.el6.x86_64
Comment 10 Qunfang Zhang 2010-06-24 02:38:31 EDT
Test on kvm-83-164.el5_5.12, kernel:2.6.18-194.3.1.el5

with the same steps, and this issue does not exist.
Comment 11 Qunfang Zhang 2011-01-11 05:07:15 EST
Verified on:
# uname  -r
2.6.18-233.el5
# rpm -q kvm
kvm-83-217.el5

This issue does not exist, so I will change the status to VERIFIED.
It's a little weird to change a bug from ON_QA to VERIFIED status without a fixed version mentioned. :-)
Maybe the code is fixed in other issue?
Comment 12 Ludek Smid 2011-01-14 09:04:08 EST
comment #10 and comment #11 indicates that issue is fixed in RHEL 5.6.

Note You need to log in before you can comment on or make changes to this bug.