Bug 1458725

Hui qing,
I am not sure if it is a libvirt bug or a qemu bug. Could you please try it with raw qemu command? Thks
As I know, this bug can be reproduced on these migration combinations:
RHEL7.4 --> RHEL7.4
RHEL7.3 --> RHEL7.3
RHEL7.3 --> RHEL7.4

Based on bug 1458699, are you sure that both the raw source and destination files (the nbd.qcow2 file being served raw by qemu-nbd) are sufficiently pre-allocated enough that the migration is not trying to write qcow2 data beyond the end of the file?

I test 7.3.z->7.4, the guest is rhel7.4 and image format is raw. 

7.3.z host:
kernel-3.10.0-514.25.2.el7.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.10.x86_64

7.4 host:
kernel-3.10.0-663.el7.x86_64
qemu-kvm-rhev-2.9.0-8.el7.x86_64

Test Steps:
1. Start nbd server as comment #0
# qemu-nbd -t -x nbd_raw --format=raw -p 30001 /home/kvm_autotest_root/images/rhel74-64-virtio.raw

2. Boot guest on source 7.3.z host:
/usr/libexec/qemu-kvm \
-machine pc-q35-rhel7.3.0,accel=kvm,usb=off,vmport=off \
-m 1024 \
-realtime mlock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid a94868ef-00d4-4f19-975c-900360c47bfc \
-no-user-config \
-nodefaults \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=discard \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot strict=on \
-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \
-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \
-device pci-bridge,chassis_nr=4,id=pci.4,bus=pci.3,addr=0x0 \
-device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \
-device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x4 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x3 \
-drive file=nbd:10.66.9.152:30001:exportname=nbd_raw,format=raw,if=none,id=drive-virtio-disk0 \
-device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-netdev tap,id=hostnet0,script=/etc/qemu-ifup \
-device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:89:de:fc,bus=pci.2,addr=0x1 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev spicevmc,id=charchannel0,name=vdagent \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \
-spice port=5900,addr=0.0.0.0,disable-ticketing,seamless-migration=on \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pcie.0,addr=0x1 \
-device intel-hda,id=sound0,bus=pci.2,addr=0x2 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-chardev spicevmc,id=charredir0,name=usbredir \
-device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=1 \
-chardev spicevmc,id=charredir1,name=usbredir \
-device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x5 \
-msg timestamp=on \
-monitor stdio

3. Boot guest with "-incoming tcp:0:5800" on 7.4 host

Results:
after step2, the guest can boot and check "dmesg", no error info.
after step3, qemu-kvm process hang:
# strace -f -p $(pidof qemu-kvm)
strace: Process 17349 attached with 2 threads
[pid 17350] futex(0x55eaa270e140, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 17349] recvmsg(18,

According to comment4, change to qemu-kvm-rhev

It maybe a dup, Refer:
Bug 1459801 - nbd/server.c:nbd_receive_request():L706: read failed when do migration

(In reply to Han Han from comment #0)

> Steps to Reproduce:
> 1. Prepare a nbd server:
> # qemu-nbd -t -x nbd_raw --format=qcow2 nbd.qcow2 -p 10803
> 2. Start a VM with nbd block like following:
> <disk type='network' device='disk'>
>       <driver name='qemu' type='qcow2'/>
>       <source protocol='nbd' name='nbd_qcow2'>
>         <host name='xx.xx.xx.xx' port='10802'/>

Is there a mismatch between the port serving the NBD image (10803) and the libvirt XML (10802), or is there a transcription error in the bug report?

(In reply to Eric Blake from comment #8)
> (In reply to Han Han from comment #0)
> 
> > Steps to Reproduce:
> > 1. Prepare a nbd server:
> > # qemu-nbd -t -x nbd_raw --format=qcow2 nbd.qcow2 -p 10803
> > 2. Start a VM with nbd block like following:
> > <disk type='network' device='disk'>
> >       <driver name='qemu' type='qcow2'/>
> >       <source protocol='nbd' name='nbd_qcow2'>
> >         <host name='xx.xx.xx.xx' port='10802'/>
> 
> Is there a mismatch between the port serving the NBD image (10803) and the
> libvirt XML (10802), or is there a transcription error in the bug report?

Well, it's a transcription error there.

Seeing manual of qemu-nbd:
       -e, --shared=num
           Allow up to num clients to share the device (default 1)


Let's set shared to 2 and try migration:
# qemu-nbd -t -p 10809 -e 2 --format=raw /var/lib/libvirt/images/PC.qcow2 -v

# virsh migrate pc qemu+ssh://10.66.4.183/system --verbose --unsafe
Migration: [100 %]  

Migration did succeed :). Obviously, the former migration failure is due to the default shared connection limits 1