Bug 771592 - Install RHEV-H to virtual machine cause VM kernel panic when boot
Install RHEV-H to virtual machine cause VM kernel panic when boot
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
5.7
x86_64 Linux
high Severity high
: rc
: 5.7
Assigned To: Paolo Bonzini
Virtualization Bugs
: Regression, ZStream
: 635251 772182 (view as bug list)
Depends On:
Blocks: 752386 772578 773322 782064 784658
  Show dependency treegraph
 
Reported: 2012-01-04 04:04 EST by shaochen
Modified: 2016-04-26 10:59 EDT (History)
22 users (show)

See Also:
Fixed In Version: kernel-2.6.18-304.el5
Doc Type: Bug Fix
Doc Text:
SG_IO ioctls were not implemented correctly in the Red Hat Enterprise Linux 5 virtio-blk driver. Sending an SG_IO ioctl request to a virtio-blk disk caused the sending thread to enter an uninterruptible sleep state ("D" state). With this update, SG_IO ioctls are rejected by the virtio-blk driver: the ioctl system call will simply return an ENOTTY ("Inappropriate ioctl for device") error and the thread will continue normally.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-02-20 23:04:43 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
kernel panic 1 (75.62 KB, image/jpeg)
2012-01-04 04:05 EST, shaochen
no flags Details
kernel panic 2 (86.35 KB, image/jpeg)
2012-01-04 04:05 EST, shaochen
no flags Details
serial log (13.84 KB, text/plain)
2012-01-04 11:20 EST, Mike Burns
no flags Details
guest kernel panic shapshot (16.90 KB, image/png)
2012-01-09 23:40 EST, FuXiangChun
no flags Details

  None (edit)
Description shaochen 2012-01-04 04:04:38 EST
Description of problem:
Create VM and install RHEV-H, the VM will kernel panic when boot.

Version-Release number of selected component (if applicable):
rhev-hypervisor-5.7-20120103.1.el5_7

How reproducible:
100%

Steps to Reproduce:
1. Register RHEV-H to RHEV-M.
2. Create storage and create VM.
3. Install RHEV-H 5.7-20120103.1.el5_7 to VM(no such issue when install other RHEV-H version).
  
Actual results:
The VM kernel panic when boot.

Expected results:
Install RHEV-H successful no kernel panic.

Additional info:
I know this isn't a correct operation, but there is no such issue when install other RHEV-H version to VM.

I am not sure which component should be selected, so I add this bug to ovirt-node.
Comment 1 shaochen 2012-01-04 04:05:08 EST
Created attachment 550635 [details]
kernel panic 1
Comment 2 shaochen 2012-01-04 04:05:53 EST
Created attachment 550636 [details]
kernel panic 2
Comment 5 Alan Pevec 2012-01-04 10:25:24 EST
Which host OS is that?
Comment 6 Paolo Bonzini 2012-01-04 11:09:19 EST
Please send the qemu-kvm commandline, too.

Do you have a vmcore?
Comment 7 Mike Burns 2012-01-04 11:20:04 EST
Created attachment 550718 [details]
serial log

Attached is the serial console log.

Below is the qemu command line

* Note that this is running on a F14 host.

qemu     18592     1 59 11:13 ?        00:01:00 /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name oni-virtio -uuid 9a2fcf21-04e9-3737-6c3c-85011ce1a90c -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/oni-virtio.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot order=dc,menu=off -device lsi,id=scsi0,bus=pci.0,addr=0x7 -drive file=/home/bos/mburns/Download/rhevh-5.7-20120103.1.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/var/lib/libvirt/images/oni-virtio.img,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/oni-virtio-1.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1 -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:c7:c4:5d,bus=pci.0,addr=0x9 -net tap,fd=48,vlan=0,name=hostnet0 -device virtio-net-pci,vlan=1,id=net1,mac=52:54:00:14:13:fe,bus=pci.0,addr=0x3 -net tap,fd=51,vlan=1,name=hostnet1 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 0.0.0.0:4 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
Comment 9 Paolo Bonzini 2012-01-04 11:23:15 EST
Mine.  It's a typo in drivers/block/virtio_blk.c
Comment 12 RHEL Product and Program Management 2012-01-05 11:10:29 EST
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 13 FuXiangChun 2012-01-05 23:53:05 EST
This are two testing scenarios
1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then can reproduce this bug. 
2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5 then guest work well.  

question:
although rhevh kernel version is the same as brewweb kernel. but file content of rhevh kernel is different from brewweb kernel, right?
Comment 14 shaochen 2012-01-06 00:32:18 EST
(In reply to comment #13)
> This are two testing scenarios
> 1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install
> guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then
> can reproduce this bug. 
> 2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5
> then guest work well.  
> 
> question:
> although rhevh kernel version is the same as brewweb kernel. but file content
> of rhevh kernel is different from brewweb kernel, right?

They are the same version.
you can check the manifest-rpm.txt.
#mount -oloop rhevh-5.7-20120103.1.el5.iso /mnt
#/vim /mnt/isolinux/manifest-rpm.txt
Comment 15 Mike Burns 2012-01-06 08:38:52 EST
(In reply to comment #14)
> (In reply to comment #13)
> > This are two testing scenarios
> > 1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install
> > guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then
> > can reproduce this bug. 
> > 2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5
> > then guest work well.  
> > 
> > question:
> > although rhevh kernel version is the same as brewweb kernel. but file content
> > of rhevh kernel is different from brewweb kernel, right?
> 
> They are the same version.
> you can check the manifest-rpm.txt.
> #mount -oloop rhevh-5.7-20120103.1.el5.iso /mnt
> #/vim /mnt/isolinux/manifest-rpm.txt

Yes, they are the same exact version in both RHEV-H and RHEL.  The only difference is that we do some blacklisting to remove unnecessary files from RHEV-H, but the actual file versions are identical.
Comment 16 Laszlo Ersek 2012-01-06 08:59:54 EST
*** Bug 772182 has been marked as a duplicate of this bug. ***
Comment 17 shaochen 2012-01-09 02:05:24 EST
Test version:
rhev-hypervisor-5.7-20120106.0.el5_7
kernel-2.6.18-274.17.1.el5

I test this version with same steps, it works correctly with no panic now.
Comment 19 juzhang 2012-01-09 20:52:11 EST
According to comment0 and comment 13,this issue is found in rhel5.7.z product.
According to comment18,the patch is for rhel5.8,do we need to open a .z bug track this issue?
Comment 20 FuXiangChun 2012-01-09 23:38:55 EST
reproduce with RHEL kernel initrd-2.6.18-274.16.1.el5.img and vmlinuz-2.6.18-274.16.1.el5

testing summary: boot guest install with RHEL kernel files and cover two disk interfaces virtio and ide. guest kernel panic when driver initialization. 

steps to reproduce:

1. in rhel6.3 host

 /usr/libexec/qemu-kvm  -M rhel6.2.0 -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -name rhel5.7z -uuid 9a2fcf21-04e7-3737-6c3c-85011ce1a90c -nodefconfig -nodefaults -rtc base=utc -boot order=n,menu=off -drive file=/home/rhel5.7-z.raw,if=none,id=drive-virtio-disk0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-virtio-disk0,id=virtio-disk0  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:c7:c5:6d,bus=pci.0,addr=0x9 -net tap,script=/etc/qemu-ifup,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb  -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -spice disable-ticketing,port=5913 -vga qxl -monitor stdio  -kernel /root/vmlinuz-2.6.18-274.16.1.el5 -initrd /root/initrd-2.6.18-274.16.1.el5.img

2. in rhel5.8 host

/usr/libexec/qemu-kvm -no-hpet -no-kvm-pit-reinjection -usbdevice tablet -rtc-td-hack -startdate 2011-12-30T07:58:33 -name rhel5.7-x86_64 -smp 2,cores=2 -k en-us -m 2048 -boot n -net nic,vlan=1,macaddr=00:1a:4a:42:0b:05,model=virtio -net tap,vlan=1,ifname=virtio_10_1,script=/etc/qemu-ifup -drive file=/home/rhe5.7z.raw,media=disk,if=ide,cache=off,serial=3a-a447-3e63d0c77247,boot=on,format=raw,werror=stop  -cpu qemu64,+sse2,+cx16,+ssse3 -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=RedHat,product=RHEL,version=5Server-5.8.0.1,serial=80BCF965-C737-E011-BF57-C145725DCBD8_1c:6f:65:06:bc:81,uuid=b4134b30-9b38-46a9-9848-ecbfb10b460f -monitor stdio -spice host=0,ic=on,port=5912,disable-ticketing -qxl 1 -kernel /root/vmlinuz-2.6.18-274.16.1.el5 -initrd /root/initrd-2.6.18-274.16.1.el5.img 

actual result:
guest kernel panic, kernel panic snapshot is in attachment.
Comment 21 FuXiangChun 2012-01-09 23:40:29 EST
Created attachment 551757 [details]
guest kernel panic shapshot
Comment 23 Jarod Wilson 2012-01-11 09:18:59 EST
Patch(es) available in kernel-2.6.18-304.el5
You can download this test kernel (or newer) from http://people.redhat.com/jwilson/el5/
Detailed testing feedback is always welcomed.
If you require guidance regarding testing, please ask the bug assignee.
Comment 27 FuXiangChun 2012-01-12 04:34:43 EST
Sorry, I need to correct a mistake, snapshot is wrong in comment21. 

can reproduce this bug with virtio disk interface and kernel 2.6.18-274.16.1.el5. snapshots are the same as comment1 and comment2.
Comment 32 Lei Wang 2012-01-30 01:24:37 EST
Move to VERIFIED per comment 28 and 31.
Comment 33 Martin Prpic 2012-02-10 07:56:59 EST
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
SG_IO ioctls were not implemented correctly in the Red Hat Enterprise
Linux 5 virtio-blk driver. Sending an SG_IO ioctl request to a virtio-blk
disk caused the sending thread to enter an uninterruptible sleep state ("D"
state). With this update, SG_IO ioctls are rejected by the virtio-blk
driver: the ioctl system call will simply return an ENOTTY ("Inappropriate
ioctl for device") error and the thread will continue normally.
Comment 34 errata-xmlrpc 2012-02-20 23:04:43 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0150.html
Comment 35 Paolo Bonzini 2012-04-16 03:03:47 EDT
*** Bug 635251 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.