Bug 1450375

Summary: Migration failed with postcopy enabled from rhel7.3.z host to rhel7.4 host "error while loading state for instance 0x0 of device 'pci@800000020000000:05.0/virtio-rng'"
Product: Red Hat Enterprise Linux 7 Reporter: Jaroslav Reznik <jreznik>
Component: qemu-kvm-rhevAssignee: Laurent Vivier <lvivier>
Status: CLOSED ERRATA QA Contact: xianwang <xianwang>
Severity: high Docs Contact:
Priority: high    
Version: 7.4CC: chayang, dgibson, dgilbert, hhuang, huding, juzhang, knoel, lvivier, michen, mrezanin, mtessun, qzhang, stefanha, virt-maint, xfu, xianwang, zhengtli
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.6.0-28.el7_3.10 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1435521 Environment:
Last Closed: 2017-06-13 07:17:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1435521    
Bug Blocks:    

Description Jaroslav Reznik 2017-05-12 11:29:00 UTC
This bug has been copied from bug #1435521 and has been proposed
to be backported to 7.3 z-stream (EUS).

Comment 3 Miroslav Rezanina 2017-05-22 09:37:47 UTC
Fix included in qemu-kvm-rhev-2.6.0-28.el7_3.10

Comment 5 xianwang 2017-05-24 10:07:29 UTC
subject is as the #comment 39 of bug1435521, huiqing, please help to check this bug for x86_64, thanks

Comment 6 xianwang 2017-05-24 10:26:57 UTC
This bug is verify pass for powerpc both on qemu-kvm-rhev-2.6.0-28.el7_3.10.ppc64le and qemu-kvm-rhev-2.9.0-2.el7.ppc64le. 
Bug verification is as below:
Host version:
(host1)
RHEL-7.3-20161019.0 Server ppc64le
kernel-3.10.0-514.17.1.el7
qemu-kvm-rhev-2.6.0-28.el7_3.10.ppc64le
SLOF-20160223-6.gitdbbfda4.el7.noarch
(host2)
RHEL-7.4-20170519.n.0 Server ppc64le
3.10.0-623.el7.ppc64le
qemu-kvm-rhev-2.9.0-2.el7.ppc64le
SLOF-20170303-4.git66d250e.el7.noarch

Guest:
RHEL-7.3-20161019.0-Server-ppc64le-dvd1.iso
3.10.0-514.21.1.el7.ppc64le
mem: 4G
vcpu: 4

scenario I:
1)steps are same with bug description.
2)result:
migration completed and vm works well.
I have test this scenario for 20 times, and all the result are pass.

scenario II:
Refering to comment 32, I have changed the qemu cli to boot guest:
/usr/libexec/qemu-kvm \
    -name 'vm73z'  \
    -sandbox off  \
    -machine pseries-rhel7.3.0 \
    -nodefaults  \
    -vga std \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial0,server,nowait \
    -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
    -device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
    -chardev socket,path=/tmp/virtio_port-vs-20170207-030401-FfusAC1v,nowait,id=idQdLRHP,server \
    -device virtserialport,id=idBu8FQH,name=vs,bus=virtio_serial_pci0.0,chardev=idQdLRHP \
    -object rng-random,filename=/dev/random,id=passthrough-rOXjKxaC \
    -device virtio-rng-pci,id=virtio-rng-pci-GVn8yzUA,rng=passthrough-rOXjKxaC,bus=pci.0,addr=05 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=07 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=unsafe,format=qcow2,file=/root/rhel73-ppc64le-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,bus=virtio_scsi_pci0.0,bootindex=0 \
    -device virtio-net-pci,mac=9a:4f:50:51:52:53,id=id9HRc5V,vectors=4,netdev=idjlQN53,bus=pci.0,addr=08  \
    -netdev tap,id=idjlQN53,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
    -m 4096 \
    -smp 8,cores=1,threads=1,sockets=8 \
    -object memory-backend-ram,policy=bind,id=mem-0,size=1024M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-0,cpus=0,cpus=1 \
    -object memory-backend-ram,policy=bind,id=mem-1,size=1024M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-1,cpus=2-3 \
    -object memory-backend-ram,policy=bind,id=mem-2,size=2048M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-2,cpus=4-7 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -device usb-mouse,id=input1,bus=usb1.0,port=2 \
    -device usb-kbd,id=input2,bus=usb1.0,port=3 \
    -vnc :1  \
    -qmp tcp:0:8881,server,nowait \
    -monitor stdio \
    -rtc base=utc,clock=host  \
    -boot order=cdn,once=c,menu=on,strict=on  \
    -enable-kvm \
    -watchdog i6300esb \
    -watchdog-action reset \
    -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=09
other steps are same with scenario I.
result:
migration completed and vm works well.

So, this bug is fixed for ppc , huiqing, could you help to verify this bug for x86_64? thanks

Comment 7 huiqingding 2017-05-25 02:31:36 UTC
Test 7.3.z->7.4 postcopy migration as comment #32, migration can be finished. 
7.3.z host:
kernel-3.10.0-514.24.1.el7.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.10.x86_64

7.4 host:
kernel-3.10.0-671.el7.x86_64
qemu-kvm-rhev-2.9.0-6.el7.x86_64

The command line is as following:
/usr/libexec/qemu-kvm \
-name rhel7 \
-S -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off \
-realtime mlock=off \
-cpu SandyBridge,check \
-sandbox off \
-m 7680   \
-smp 8,cores=1,threads=1,sockets=8  \
-object memory-backend-ram,policy=bind,id=mem-0,size=512M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-0,cpus=0,cpus=1 \
-object memory-backend-ram,policy=bind,id=mem-1,size=1024M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-1,cpus=2-3 \
-object memory-backend-ram,policy=bind,id=mem-2,size=2048M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-2,cpus=4 \
-object memory-backend-ram,policy=bind,id=mem-3,size=4096M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-3,cpus=5-7 \
-uuid 49a3438a-70a3-4ba8-92ce-3a05e0934608 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel7.monitor,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=discard \
-no-hpet \
-no-shutdown \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot order=c,menu=on,splash-time=3000,strict=on \
-device ich9-usb-ehci1,id=usb0,bus=pci.0,addr=0x5.0x7 \
-device ich9-usb-uhci1,masterbus=usb0.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 \
-device ich9-usb-uhci2,masterbus=usb0.0,firstport=2,bus=pci.0,addr=0x5.0x1 \
-device ich9-usb-uhci3,masterbus=usb0.0,firstport=4,bus=pci.0,addr=0x5.0x2 \
-device virtio-scsi-pci,id=scsi0,cmd_per_lun=234,bus=pci.0,addr=0x8 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \
-drive file=/usr/share/virtio-win/virtio-win-1.9.0.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw,serial= \
-device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
-drive file=/usr/share/virtio-win/virtio-win_x86.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none \
-global isa-fdc.driveA=drive-fdc0-0-0 \
-drive file=/mnt/stable_guest_abi/rhel74-64-virtio.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=f65effa5-90a6-47f2-8487-a9f64c95d4f5,cache=none,discard=unmap,werror=stop,rerror=stop,aio=threads \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-drive file=/mnt/virtio-scsi-disk,if=none,id=drive-scsi-disk,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi1,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi1.0,id=data-disk2 \
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 \
-chardev spicevmc,id=charchannel0,name=vdagent \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \
-chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channel/target/rhel7.org.qemu.guest_agent.0,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0 \
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-device intel-hda,id=sound1,bus=pci.0,addr=0x6 \
-device hda-micro,id=sound1-codec0,bus=sound1.0 \
-device intel-hda,id=sound2,bus=pci.0,addr=0x17 \
-device hda-output,id=sound2-codec0,bus=sound2.0,cad=0 \
-device ich9-intel-hda,id=sound3,bus=pci.0,addr=0x18 \
-device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0 \
-chardev spicevmc,id=charredir0,name=usbredir \
-device usb-redir,chardev=charredir0,id=redir0 \
-chardev spicevmc,id=charredir1,name=usbredir \
-device usb-redir,chardev=charredir1,id=redir1 \
-chardev spicevmc,id=charredir2,name=usbredir \
-device usb-redir,chardev=charredir2,id=redir2 \
-chardev spicevmc,id=charredir3,name=usbredir \
-device usb-redir,chardev=charredir3,id=redir3 \
-device usb-host,id=hostdev0 \
-device pvpanic,ioport=1285 -msg timestamp=on \
-netdev tap,id=hostnet0,vhost=on,id=hostnet0,script=/etc/qemu-ifup \
-device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=b6:af:42:c8:66:18,bus=pci.0,addr=0x14 \
-netdev tap,id=hostnet1,vhost=on,script=/etc/qemu-ifup  \
-device e1000,netdev=hostnet1,id=virtio-net-pci1,mac=b6:2f:a8:85:82:7c,bus=pci.0,addr=0x15,multifunction=off \
-netdev tap,id=hostnet2,vhost=on,script=/etc/qemu-ifup \
-device rtl8139,netdev=hostnet2,id=virtio-net-pci2,mac=4e:63:28:bc:c1:75,bus=pci.0,addr=0x16,multifunction=off \
-netdev tap,id=hostnet3,vhost=on,script=/etc/qemu-ifup \
-device e1000-82540em,netdev=hostnet3,id=virtio-net-pci3,mac=4e:63:28:bc:c1:65,bus=pci.0,addr=0x1e,multifunction=off \
-netdev tap,id=hostnet4,vhost=on,script=/etc/qemu-ifup \
-device e1000e,netdev=hostnet4,id=virtio-net-pci4,mac=4e:63:28:bc:c1:85,bus=pci.0,addr=0x19,multifunction=off \
-drive file=/mnt/ide-disk,if=none,id=drive-data-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop,copy-on-read=off,serial=fux-ide,media=disk \
-device ide-hd,drive=drive-data-disk,id=system-disk,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,ver=fuxc-ver,bus=ide.0,unit=0  \
-device ich9-usb-uhci6,id=uhci6,bus=pci.0,addr=0xa \
-device usb-kbd,id=kdb0,bus=uhci6.0 \
-device ich9-usb-uhci5,id=uhci5,bus=pci.0,addr=0xb \
-device usb-mouse,id=mouse0,bus=uhci5.0 \
-device nec-usb-xhci,id=xhci,bus=pci.0,addr=0xd \
-device usb-bot,id=bot1,bus=xhci.0 \
-drive file=/mnt/virtio-win-126.iso,if=none,id=usb-cdrom1,format=raw \
-device scsi-cd,bus=bot1.0,scsi-id=0,lun=1,drive=usb-cdrom1,id=usb-cdrom1 \
-drive file=/mnt/bot-disk1,id=usb-disk1,if=none,format=qcow2 \
-device scsi-hd,bus=bot1.0,scsi-id=0,lun=0,drive=usb-disk1,id=usb-disk1 \
-device usb-ehci,id=ehci,bus=pci.0,addr=0xe \
-device usb-bot,id=bot2,bus=ehci.0 \
-drive file=/mnt/virtio-win-126.iso,if=none,id=usb-cdrom2,format=raw \
-device scsi-cd,bus=bot2.0,scsi-id=0,lun=1,drive=usb-cdrom2,id=usb-cdrom2 \
-drive file=/mnt/bot-disk2,id=usb-disk2,if=none,format=qcow2 \
-device scsi-hd,bus=bot2.0,scsi-id=0,lun=0,drive=usb-disk2,id=usb-disk2 \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0xf \
-device usb-bot,id=bot3,bus=usb.0 \
-drive file=/mnt/virtio-win-126.iso,if=none,id=usb-cdrom3,format=raw \
-device scsi-cd,bus=bot3.0,scsi-id=0,lun=1,drive=usb-cdrom3,id=usb-cdrom3 \
-drive file=/mnt/bot-disk3,id=usb-disk3,if=none,format=qcow2 \
-device scsi-hd,bus=bot3.0,scsi-id=0,lun=0,drive=usb-disk3,id=usb-disk3 \
-device ich9-usb-uhci3,id=uhci,bus=pci.0,addr=0x10 \
-device usb-storage,drive=drive-usb-0,id=usb-0,removable=on,bus=uhci.0,port=1 \
-drive file=/mnt/usb-uhci,if=none,id=drive-usb-0,media=disk,format=qcow2 \
-device ich9-usb-ehci1,id=ehci1,bus=pci.0,addr=0x11 \
-device usb-storage,drive=drive-usb-1,id=usb-1,removable=on,bus=ehci1.0,port=1 \
-drive file=/mnt/usb-ehci,if=none,id=drive-usb-1,media=disk,format=qcow2 \
-device nec-usb-xhci,id=xhci1,bus=pci.0,addr=0x12 \
-device usb-storage,drive=drive-usb-2,id=usb-2,removable=on,bus=xhci1.0,port=1 \
-drive file=/mnt/usb-xhci,if=none,id=drive-usb-2,media=disk,format=qcow2 \
-watchdog ib700 -watchdog-action reset \
-device virtio-rng-pci,id=rng0,bus=pci.0,addr=0x1c \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1d \
-monitor stdio \
-qmp tcp:0:4466,server,nowait -serial unix:/tmp/ttym,server,nowait \
-spice port=5910,addr=0.0.0.0,disable-ticketing,seamless-migration=on \
-k en-us \
-device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 \
-chardev socket,id=seabioslog_id,path=/var/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabioslog_id,iobase=0x402 \

Comment 8 huiqingding 2017-05-25 02:32:50 UTC
Based on comment #6 and comment #7, set this bug to be verified.

Comment 10 errata-xmlrpc 2017-06-13 07:17:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1431