Bug 771592

Summary:

Install RHEV-H to virtual machine cause VM kernel panic when boot

Product:

Red Hat Enterprise Linux 5

Reporter:

cshao <cshao>

Component:

kernel

Assignee:

Paolo Bonzini <pbonzini>

Status:

CLOSED ERRATA

QA Contact:

Virtualization Bugs <virt-bugs>

Severity:

high

Docs Contact:

Priority:

high

Version:

5.7

CC:

apevec, areis, bsarathy, dhoward, emcnabb, gouyang, jboggs, jpallich, juzhang, leiwang, mburns, michen, moli, ovirt-maint, pbonzini, pmatouse, qcai, rommer, sforsber, syeghiay, xfu, ycui

Target Milestone:

Keywords:

Regression, ZStream

Target Release:

5.7

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

kernel-2.6.18-304.el5

Doc Type:

Bug Fix

Doc Text:

SG_IO ioctls were not implemented correctly in the Red Hat Enterprise Linux 5 virtio-blk driver. Sending an SG_IO ioctl request to a virtio-blk disk caused the sending thread to enter an uninterruptible sleep state ("D" state). With this update, SG_IO ioctls are rejected by the virtio-blk driver: the ioctl system call will simply return an ENOTTY ("Inappropriate ioctl for device") error and the thread will continue normally.

Story Points:

---

Clone Of:

Environment:

Last Closed:

2012-02-21 04:04:43 UTC

Type:

---

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

752386, 772578, 773322, 782064, 784658

Attachments:

Description	Flags
kernel panic 1	none
kernel panic 2	none
serial log	none
guest kernel panic shapshot	none

Description cshao 2012-01-04 09:04:38 UTC

Description of problem:
Create VM and install RHEV-H, the VM will kernel panic when boot.

Version-Release number of selected component (if applicable):
rhev-hypervisor-5.7-20120103.1.el5_7

How reproducible:
100%

Steps to Reproduce:
1. Register RHEV-H to RHEV-M.
2. Create storage and create VM.
3. Install RHEV-H 5.7-20120103.1.el5_7 to VM(no such issue when install other RHEV-H version).
  
Actual results:
The VM kernel panic when boot.

Expected results:
Install RHEV-H successful no kernel panic.

Additional info:
I know this isn't a correct operation, but there is no such issue when install other RHEV-H version to VM.

I am not sure which component should be selected, so I add this bug to ovirt-node.

Comment 1 cshao 2012-01-04 09:05:08 UTC

Created attachment 550635 [details]
kernel panic 1

Comment 2 cshao 2012-01-04 09:05:53 UTC

Created attachment 550636 [details]
kernel panic 2

Comment 5 Alan Pevec 2012-01-04 15:25:24 UTC

Which host OS is that?

Comment 6 Paolo Bonzini 2012-01-04 16:09:19 UTC

Please send the qemu-kvm commandline, too.

Do you have a vmcore?

Comment 7 Mike Burns 2012-01-04 16:20:04 UTC

Created attachment 550718 [details]
serial log

Attached is the serial console log.

Below is the qemu command line

* Note that this is running on a F14 host.

qemu     18592     1 59 11:13 ?        00:01:00 /usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name oni-virtio -uuid 9a2fcf21-04e9-3737-6c3c-85011ce1a90c -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/oni-virtio.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot order=dc,menu=off -device lsi,id=scsi0,bus=pci.0,addr=0x7 -drive file=/home/bos/mburns/Download/rhevh-5.7-20120103.1.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/var/lib/libvirt/images/oni-virtio.img,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/oni-virtio-1.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1 -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:c7:c4:5d,bus=pci.0,addr=0x9 -net tap,fd=48,vlan=0,name=hostnet0 -device virtio-net-pci,vlan=1,id=net1,mac=52:54:00:14:13:fe,bus=pci.0,addr=0x3 -net tap,fd=51,vlan=1,name=hostnet1 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 0.0.0.0:4 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

Comment 9 Paolo Bonzini 2012-01-04 16:23:15 UTC

Mine.  It's a typo in drivers/block/virtio_blk.c

Comment 12 RHEL Program Management 2012-01-05 16:10:29 UTC

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 13 FuXiangChun 2012-01-06 04:53:05 UTC

This are two testing scenarios
1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then can reproduce this bug. 
2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5 then guest work well.  

question:
although rhevh kernel version is the same as brewweb kernel. but file content of rhevh kernel is different from brewweb kernel, right?

Comment 14 cshao 2012-01-06 05:32:18 UTC

(In reply to comment #13)
> This are two testing scenarios
> 1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install
> guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then
> can reproduce this bug. 
> 2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5
> then guest work well.  
> 
> question:
> although rhevh kernel version is the same as brewweb kernel. but file content
> of rhevh kernel is different from brewweb kernel, right?

They are the same version.
you can check the manifest-rpm.txt.
#mount -oloop rhevh-5.7-20120103.1.el5.iso /mnt
#/vim /mnt/isolinux/manifest-rpm.txt

Comment 15 Mike Burns 2012-01-06 13:38:52 UTC

(In reply to comment #14)
> (In reply to comment #13)
> > This are two testing scenarios
> > 1.If using rhevh-5.7-20120103.1.iso(kernel 2.6.18-274.16.1.el5) to install
> > guest or use gpxe(profile rhev-hypervisor-5.7-20120103.1) to install guest then
> > can reproduce this bug. 
> > 2.if installing a GA rhel5.7 guest and update kernel to 2.6.18-274.16.1.el5
> > then guest work well.  
> > 
> > question:
> > although rhevh kernel version is the same as brewweb kernel. but file content
> > of rhevh kernel is different from brewweb kernel, right?
> 
> They are the same version.
> you can check the manifest-rpm.txt.
> #mount -oloop rhevh-5.7-20120103.1.el5.iso /mnt
> #/vim /mnt/isolinux/manifest-rpm.txt

Yes, they are the same exact version in both RHEV-H and RHEL.  The only difference is that we do some blacklisting to remove unnecessary files from RHEV-H, but the actual file versions are identical.

Comment 16 Laszlo Ersek 2012-01-06 13:59:54 UTC

*** Bug 772182 has been marked as a duplicate of this bug. ***

Comment 17 cshao 2012-01-09 07:05:24 UTC

Test version:
rhev-hypervisor-5.7-20120106.0.el5_7
kernel-2.6.18-274.17.1.el5

I test this version with same steps, it works correctly with no panic now.

Comment 19 juzhang 2012-01-10 01:52:11 UTC

According to comment0 and comment 13,this issue is found in rhel5.7.z product.
According to comment18,the patch is for rhel5.8,do we need to open a .z bug track this issue?

Comment 20 FuXiangChun 2012-01-10 04:38:55 UTC

reproduce with RHEL kernel initrd-2.6.18-274.16.1.el5.img and vmlinuz-2.6.18-274.16.1.el5

testing summary: boot guest install with RHEL kernel files and cover two disk interfaces virtio and ide. guest kernel panic when driver initialization. 

steps to reproduce:

1. in rhel6.3 host

 /usr/libexec/qemu-kvm  -M rhel6.2.0 -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -name rhel5.7z -uuid 9a2fcf21-04e7-3737-6c3c-85011ce1a90c -nodefconfig -nodefaults -rtc base=utc -boot order=n,menu=off -drive file=/home/rhel5.7-z.raw,if=none,id=drive-virtio-disk0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-virtio-disk0,id=virtio-disk0  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:c7:c5:6d,bus=pci.0,addr=0x9 -net tap,script=/etc/qemu-ifup,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb  -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -spice disable-ticketing,port=5913 -vga qxl -monitor stdio  -kernel /root/vmlinuz-2.6.18-274.16.1.el5 -initrd /root/initrd-2.6.18-274.16.1.el5.img

2. in rhel5.8 host

/usr/libexec/qemu-kvm -no-hpet -no-kvm-pit-reinjection -usbdevice tablet -rtc-td-hack -startdate 2011-12-30T07:58:33 -name rhel5.7-x86_64 -smp 2,cores=2 -k en-us -m 2048 -boot n -net nic,vlan=1,macaddr=00:1a:4a:42:0b:05,model=virtio -net tap,vlan=1,ifname=virtio_10_1,script=/etc/qemu-ifup -drive file=/home/rhe5.7z.raw,media=disk,if=ide,cache=off,serial=3a-a447-3e63d0c77247,boot=on,format=raw,werror=stop  -cpu qemu64,+sse2,+cx16,+ssse3 -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=RedHat,product=RHEL,version=5Server-5.8.0.1,serial=80BCF965-C737-E011-BF57-C145725DCBD8_1c:6f:65:06:bc:81,uuid=b4134b30-9b38-46a9-9848-ecbfb10b460f -monitor stdio -spice host=0,ic=on,port=5912,disable-ticketing -qxl 1 -kernel /root/vmlinuz-2.6.18-274.16.1.el5 -initrd /root/initrd-2.6.18-274.16.1.el5.img 

actual result:
guest kernel panic, kernel panic snapshot is in attachment.

Comment 21 FuXiangChun 2012-01-10 04:40:29 UTC

Created attachment 551757 [details]
guest kernel panic shapshot

Comment 23 Jarod Wilson 2012-01-11 14:18:59 UTC

Patch(es) available in kernel-2.6.18-304.el5
You can download this test kernel (or newer) from http://people.redhat.com/jwilson/el5/
Detailed testing feedback is always welcomed.
If you require guidance regarding testing, please ask the bug assignee.

Comment 27 FuXiangChun 2012-01-12 09:34:43 UTC

Sorry, I need to correct a mistake, snapshot is wrong in comment21. 

can reproduce this bug with virtio disk interface and kernel 2.6.18-274.16.1.el5. snapshots are the same as comment1 and comment2.

Comment 32 Lei Wang 2012-01-30 06:24:37 UTC

Move to VERIFIED per comment 28 and 31.

Comment 33 Martin Prpič 2012-02-10 12:56:59 UTC

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
SG_IO ioctls were not implemented correctly in the Red Hat Enterprise
Linux 5 virtio-blk driver. Sending an SG_IO ioctl request to a virtio-blk
disk caused the sending thread to enter an uninterruptible sleep state ("D"
state). With this update, SG_IO ioctls are rejected by the virtio-blk
driver: the ioctl system call will simply return an ENOTTY ("Inappropriate
ioctl for device") error and the thread will continue normally.

Comment 34 errata-xmlrpc 2012-02-21 04:04:43 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0150.html

Comment 35 Paolo Bonzini 2012-04-16 07:03:47 UTC

*** Bug 635251 has been marked as a duplicate of this bug. ***