Bug 1138579 - Migration failed with nec-usb-xhci from RHEL7. 0 to RHEL7.1
Summary: Migration failed with nec-usb-xhci from RHEL7. 0 to RHEL7.1
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.1
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Dr. David Alan Gilbert
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-05 08:50 UTC by huiqingding
Modified: 2015-03-05 09:54 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-rhev-2.1.0-4.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-05 09:54:39 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0624 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2015-03-05 14:37:36 UTC

Description huiqingding 2014-09-05 08:50:07 UTC
Description of problem:
Migration failed with usb storage of nec-usb-xhci from RHEL7.0 to RHEL7.1.

Version-Release number of selected component (if applicable):
src host:
qemu-kvm-rhev-1.5.3-60.el7ev.x86_64
kernel-3.10.0-123.8.1.el7.x86_64

dst host:
kernel-3.10.0-152.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. boot a rhel7.1 guest with nec-usb-xhci storage on src host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-monitor stdio

2. boot the guest on dst host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-incoming tcp:0:5800

3. do migration
(qemu) migrate -d tcp:10.66.9.152:5800

Actual results:
after step3, migration is failed and qemu-kvm quits on the dst host:
(qemu) red_dispatcher_loadvm_commands: 
qemu: warning: error while loading state for instance 0x0 of device '0000:00:12.0/xhci'
qemu-kvm: load of migration failed: Invalid argument


Expected results:
migration can be finished normally and qemu-kvm doesn't quit.

Additional info:

Comment 1 huiqingding 2014-09-05 08:53:02 UTC
I also test the following version, also hit this problem:
src host:
qemu-kvm-rhev-1.5.3-60.el7_0.7.x86_64
kernel-3.10.0-123.8.1.el7.x86_64

dst host:
kernel-3.10.0-152.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64

Comment 3 Dr. David Alan Gilbert 2014-09-08 14:25:28 UTC
Reproduced; there's a possibility that it's already fixed by an upstream fix that's getting backported at the moment, although that xhci bug is listed  as 'rare' where as this is fully repeatable.

Comment 4 Dr. David Alan Gilbert 2014-09-08 14:52:19 UTC
Apparently not fixed by the bz 1093023/1136512 fix from Laszlo/Markus

Comment 5 Dr. David Alan Gilbert 2014-09-08 15:51:25 UTC
vmstate_load_state: For xhci id=1
vmstate_load_state: field parent_obj
vmstate_load_state: For PCIEDevice id=2
vmstate_load_state: field version_id
vmstate_load_state: field config
pci.c:get_pci_config_device EINVAL for mask/flags mismatch i=113 config[i]=a0 s->config[i]=0 s->cmask[i]=ff s->wmask[i]=0 s->w1cmask[i]=0
Bailing -22 for config in PCIEDevice
Bailing -22 for parent_obj in xhci
qemu: warning: error while loading state for instance 0x0 of device '0000:00:12.0/xhci'
qemu-system-x86_64: load of migration failed: Invalid argument

Comment 6 Dr. David Alan Gilbert 2014-09-08 16:23:34 UTC
lspci -v on rhel 7.0 is showing:

lspci -v shows
  [90] MSI-X: Enable+ Count=16 Masked-
  [70] MSI: Enable- Count=1/16 Maskable- 64bit+
  [a0] Express Endpoint, MSI 00

on rhev 7.1 I'm seeing:

  [90] MSI-X: Enable+ Count=16 Masked-
  [70] MSI: Enable- Count=1/16 Maskable- 64bit+

so it looks like the Express Endpoint config has been eaten.

Comment 8 Dr. David Alan Gilbert 2014-09-09 10:29:14 UTC
posted fix upstream

Comment 10 Miroslav Rezanina 2014-09-17 07:38:17 UTC
Fix included in qemu-kvm-rhev-2.1.0-4.el7

Comment 12 huiqingding 2014-09-23 06:41:39 UTC
Reproduce this bug using the following version:
src host:
qemu-kvm-rhev-1.5.3-60.el7ev.x86_64
kernel-3.10.0-123.8.1.el7.x86_64

dst host:
kernel-3.10.0-167.el7.x86_64
qemu-kvm-rhev-2.1.0-3.el7.x86_64

Steps to Reproduce:
1. boot a rhel7.1 guest with nec-usb-xhci storage on src host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-monitor stdio

2. boot the guest on dst host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-incoming tcp:0:5800

3. do migration
(qemu) migrate -d tcp:10.66.9.152:5800

Actual results:
after step3, migration is failed and qemu-kvm quits on the dst host:
(qemu) red_dispatcher_loadvm_commands: 
qemu: warning: error while loading state for instance 0x0 of device '0000:00:12.0/xhci'
qemu-kvm: load of migration failed: Invalid argument

Comment 13 huiqingding 2014-09-23 06:45:09 UTC
Test this bug on an intel host using the following version:
src host:
qemu-kvm-rhev-1.5.3-60.el7ev.x86_64
kernel-3.10.0-123.8.1.el7.x86_64

dst host:
kernel-3.10.0-167.el7.x86_64
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Steps to Test:
1. boot a rhel7.1 guest with nec-usb-xhci storage on src host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-monitor stdio

2. boot the guest on dst host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/rhel7-ga.qcow2_v3,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-incoming tcp:0:5800

3. do migration
(qemu) migrate -d tcp:10.66.9.152:5800

Actual results:
Migration can be finished normally. Can create a file on the nec-usb-xhci storage using "dd if=/dev/zero of=file bs=1M count=1000".

Comment 14 huiqingding 2014-09-23 08:08:52 UTC
Test this bug on an amd host using the following version:
src host:
qemu-kvm-rhev-1.5.3-60.el7_0.9.x86_64
kernel-3.10.0-163.el7.x86_64

dst host:
kernel-3.10.0-167.el7.x86_64
qemu-kvm-rhev-2.1.0-4.el7.x86_64

Guest:
Windows 8.1 32 bits

Steps to Test:
1. boot a win8.1_32 guest with nec-usb-xhci storage on src host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/win8.1_32.qcow2,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-monitor stdio

2. boot the guest on dst host
# /usr/libexec/qemu-kvm -cpu SandyBridge \
-enable-kvm  -m 4096 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1,maxcpus=160 -numa node,cpus=0 \
-M pc-i440fx-rhel7.0.0 \
-drive file=/mnt/win8.1_32.qcow2,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi0,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi0.0,id=data-disk2,bootindex=1 \
-device nec-usb-xhci,id=xhci,addr=0x12 \
-device usb-storage,drive=drive-usb-0-1,id=usb-0-1,removable=on,bus=xhci.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-0-1,media=disk,format=qcow2 \
-spice port=5901,password=redhat-vga,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 \
-incoming tcp:0:5800

3. do migration
(qemu) migrate -d tcp:10.66.106.21:5800

Actual results:
Migration can be finished normally. Can create a file on the nec-usb-xhci storage.

Comment 17 errata-xmlrpc 2015-03-05 09:54:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html


Note You need to log in before you can comment on or make changes to this bug.