Bug 1320473
| Summary: | virtio-pci doesn't reset VQ correctly at device reset | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Han Han <hhan> |
| Component: | qemu-kvm-rhev | Assignee: | Fam Zheng <famz> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.3 | CC: | chayang, dyuan, famz, hhan, huding, jinzhao, juzhang, knoel, mrezanin, sherold, virt-maint, xuzhang, yama |
| Target Milestone: | rc | Keywords: | Regression |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | QEMU 2.6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-11-07 21:02:08 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The bug can also be reproduced on qemu-kvm-2.5.0-11.fc25.x86_64 qemu-kvm-rhev-2.5.0-3.el7.x86_64 Can you try with upstream QEMU? If that works, it's fixed in QEMU 2.6.
There is a reset bug fixed recently:
commit 75fd6f13af8513f1e14add754549141e415f8704
Author: Gerd Hoffmann <kraxel>
Date: Thu Jan 28 16:08:07 2016 +0100
virtio-pci: call pci reset variant when guest requests reset.
Actually fixes linux not finding virtio 1.0 device virtqueues after
reboot. Which is new I think, any chance linux kernel virtio code
became more strict in 4.3?
Signed-off-by: Gerd Hoffmann <kraxel>
Reviewed-by: Michael S. Tsirkin <mst>
Signed-off-by: Michael S. Tsirkin <mst>
Reviewed-by: Fam Zheng <famz>
Tested-by: Fam Zheng <famz>
(In reply to Fam Zheng from comment #4) > Can you try with upstream QEMU? If that works, it's fixed in QEMU 2.6. > > There is a reset bug fixed recently: > > commit 75fd6f13af8513f1e14add754549141e415f8704 > Author: Gerd Hoffmann <kraxel> > Date: Thu Jan 28 16:08:07 2016 +0100 > > virtio-pci: call pci reset variant when guest requests reset. > > Actually fixes linux not finding virtio 1.0 device virtqueues after > reboot. Which is new I think, any chance linux kernel virtio code > became more strict in 4.3? > > Signed-off-by: Gerd Hoffmann <kraxel> > Reviewed-by: Michael S. Tsirkin <mst> > Signed-off-by: Michael S. Tsirkin <mst> > Reviewed-by: Fam Zheng <famz> > Tested-by: Fam Zheng <famz> Well, I built qemu-2.6.0-rc0 and test as comment1. There is no problem. Disk is ok to be read&write with virtio1.0 on. OK, thanks for the testing! Let's wait for the 7.3 rebase to pick this up. Setting to POST therefore. Reproduce this bug using the following version: qemu-kvm-rhev-2.5.0-2.el7.x86_64 kernel-3.10.0-393.el7.x86_64 1. Boot rhel7.3 guest with virtio-blk disk, set disable-legacy=on,disable-modern=off. # /usr/libexec/qemu-kvm -name qemu,debug-threads=on -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off,vmport=off -cpu Penryn -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d597f0f5-e78f-4d58-a9f4-37527c64dc0c -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-34-qemu/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/home/qemu-1.qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=29,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:95:e1:20,bus=pci.0,addr=0x9 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channel/target/domain-34-qemu/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0 -spice port=5901,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -drive file=/tmp/haha,format=raw,if=none,id=drive-virtio-disk1 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x13,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=2,disable-legacy=true,disable-modern=false -msg timestamp=on -monitor stdio 2. Inside guest, the virtio-blk disk cannot be found. # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk ├─sda1 8:1 0 500M 0 part /boot └─sda2 8:2 0 9.5G 0 part ├─fedora-root 253:0 0 8.5G 0 lvm / └─fedora-swap 253:1 0 1G 0 lvm [SWAP] # dmesg |grep virtio -i [ 4.597846] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver [ 4.739341] virtio-pci 0000:00:08.0: virtio_pci: leaving for legacy driver [ 4.746057] virtio_blk: probe of virtio0 failed with error -2 Verify this bug using the following version: qemu-kvm-rhev-2.6.0-1.el7.x86_64 kernel-3.10.0-393.el7.x86_64 Use the commandline of comment8 to boot RHEL7.3 guest, inside guest, the virtio-blk disk can be found. # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 252:0 0 3G 0 disk ├─vda1 252:1 0 500M 0 part /boot └─vda2 252:2 0 2.5G 0 part ├─rhel-root 253:0 0 2.2G 0 lvm / └─rhel-swap 253:1 0 308M 0 lvm [SWAP] vdb 252:16 0 100M 0 disk # dmesg|grep virtio -i [ 1.717794] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver [ 1.864379] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver [ 1.903274] virtio-pci 0000:00:08.0: virtio_pci: leaving for legacy driver [ 1.927226] virtio-pci 0000:00:03.0: irq 24 for MSI/MSI-X [ 1.927241] virtio-pci 0000:00:03.0: irq 25 for MSI/MSI-X [ 1.929059] virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X [ 1.929073] virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X [ 1.931968] virtio-pci 0000:00:13.0: irq 28 for MSI/MSI-X [ 1.931982] virtio-pci 0000:00:13.0: irq 29 for MSI/MSI-X # mount /dev/vdb /mnt # cd /mnt # dd if=/dev/zero of=file bs=1M count=10 # ll file -rw-r--r--. 1 root root 10485760 May 19 16:45 file Additional info: The cpu of host is as following: # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 42 Model name: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz Stepping: 7 CPU MHz: 1599.726 BogoMIPS: 6784.95 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K NUMA node0 CPU(s): 0-7 Hi fam, I what to know qemu will enable virtio-legacy or virtio-modern or both when not setting disable-legacy and disable-modern flag?
This can be seen from the code
hw/virtio/virtio-pci.c:
DEFINE_PROP_BIT("disable-legacy", VirtIOPCIProxy, flags,
VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT, false),
DEFINE_PROP_BIT("disable-modern", VirtIOPCIProxy, flags,
VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT, true),
disable-legacy is false by default. disable-mordern is true by default. So, by default we use legacy.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2673.html |
Description of problem: For a virtio disk(non-system disk), set disable-legacy=on,disable-modern=off. The disk cannot be found in VM. Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.5.0-2.el7.x86_64 libvirt-1.3.2-1.el7.x86_64 kernel-3.10.0-330.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. Start a VM with following xml: <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/qemu-virtio-2.qcow2'/> <target dev='hda' bus='ide'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <qemu:commandline> <qemu:arg value='-drive'/> <qemu:arg value='file=/tmp/haha,format=raw,if=none,id=drive-virtio-disk0'/> <qemu:arg value='-device'/> <qemu:arg value='virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2,disable-legacy=true,disable-modern=false'/> <qemu:arg value='-monitor'/> <qemu:arg value='telnet:0.0.0.0:1234,server,nowait'/> </qemu:commandline> The hda is VM's os image. 2. Check the virtio disk: In guest, # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 10G 0 disk ├─sda1 8:1 0 500M 0 part /boot └─sda2 8:2 0 9.5G 0 part ├─fedora-root 253:0 0 8.5G 0 lvm / └─fedora-swap 253:1 0 1G 0 lvm [SWAP] # lspci 00:03.0 SCSI storage controller: Red Hat, Inc Device 1042 (rev 01) Subsystem: Red Hat, Inc Device 1100 Physical Slot: 3 Flags: bus master, fast devsel, latency 0, IRQ 11 Memory at fc056000 (32-bit, non-prefetchable) [size=4K] Memory at fe000000 (64-bit, prefetchable) [size=8M] Capabilities: [98] MSI-X: Enable- Count=2 Masked- Capabilities: [84] Vendor Specific Information: VirtIO: <unknown> Capabilities: [70] Vendor Specific Information: VirtIO: Notify Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg Capabilities: [50] Vendor Specific Information: VirtIO: ISR Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg Kernel driver in use: virtio-pci Kernel modules: virtio_pci # dmesg |grep virtio -i [ 4.597846] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver [ 4.739341] virtio-pci 0000:00:08.0: virtio_pci: leaving for legacy driver [ 4.746057] virtio_blk: probe of virtio0 failed with error -2 Actual results: as step2 Expected results: virtio1.0 disk works well. Additional info: qemu-kvm-rhev-2.3.0-31.el7_2.10.x86_64 works well. virtio0.9(disable-legacy=off disable-modern=on) is ok in qemu-kvm-rhev-2.5.0-2.el7.x86_64.