Description of problem: Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.9.0-8.el7.x86_64 How reproducible: 2/2 Steps to Reproduce: 1. Server: # qemu-nbd -f qcow2 rhel74-64-virtio.qcow2 -p 9000 -t --share=2 2. Client: Boot up guest cmd: /usr/libexec/qemu-kvm \ -S \ -name 'avocado-vt-vm1' \ -sandbox off \ -machine pc \ -nodefaults \ -vga cirrus \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_dGGu2S/monitor-qmpmonitor1-20170608-041234-CUqSlEp0,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_dGGu2S/monitor-catch_monitor-20170608-041234-CUqSlEp0,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idHoC3M4 \ -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_dGGu2S/serial-serial0-20170608-041234-CUqSlEp0,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20170608-041234-CUqSlEp0,path=/var/tmp/avocado_dGGu2S/seabios-20170608-041234-CUqSlEp0,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20170608-041234-CUqSlEp0,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pci.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pci.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pci.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pci.0 \ -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=raw,file=nbd:10.73.196.131:9000 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x3 \ -device virtio-net-pci,mac=9a:15:16:17:18:19,id=idmk5xPz,vectors=4,netdev=iddxJKCe,bus=pci.0,addr=0x4 \ -netdev tap,id=iddxJKCe,vhost=on,vhostfd=37,fd=36 \ -m 4096 \ -smp 4,cores=2,threads=1,sockets=2 \ -cpu 'Westmere',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :2 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -incoming tcp:0:5200 3. Do live migration {'execute': 'migrate', 'arguments': {'uri': 'tcp:localhost:5200', 'blk': False, 'inc': False}, 'id': 'Ig1TYidk'} Actual results: nbd/server.c:nbd_receive_request():L706: read failed error show in server guest can be migrated successfully Expected results: Additional info:
the error happen before the migration operation. boot up src image and boot up dst image, then the error show
I can reproduce with qemu-kvm-rhev-2.8.0-6.el7.x86_64 do migration cancel in src guest: src: (qemu) migrate -d tcp:0:5200 (qemu) migrate_cancel result: dst guest: (qemu) qemu-kvm: Unknown combination of migration flags: 0 qemu-kvm: error while loading state section id 3(ram) qemu-kvm: load of migration failed: Invalid argument server: # qemu-nbd -f qcow2 rhel74-64-virtio-scsi.qcow2 --share=4 -t -p 9000 nbd/server.c:nbd_receive_request():L710: read failed
not the same as Bug 1458725, as it just happen when do migrate_cancel. migration canbe do successfully if didn't cancel it.
I wonder if NBD drives should be a migration blocker? After all, having two separate clients (source and destination of the qemu migration) both trying to connect as read-write clients to the same NBD server may not always work right. The NBD protocol permits servers to allow multiple clients, but does not require it to work well, and I don't know if qemu-nbd as a server is designed to handle this sort of scenario.
Reproduced this bug in rhel8. Tested with: qemu-kvm-3.1.0-20.module+el8+2904+e658c755 kernel-4.18.0-80.el8 Steps: Server: # qemu-nbd -f qcow2 sn.qcow2 -t -p 9000 --share=2 Client: 1. Boot guest in source # /usr/libexec/qemu-kvm \ -name 'guest' \ -machine q35 \ -nodefaults \ -vga qxl \ -vnc :1 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x3 \ -blockdev driver=nbd,cache.direct=on,cache.no-flush=off,server.host=localhost,server.port=9000,server.type=inet,node-name=my_file \ -blockdev driver=raw,node-name=my,file=my_file \ -device scsi-hd,drive=my \ -monitor stdio \ -m 8192 \ -smp 8 \ -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9 \ -netdev tap,id=idxgXAlm \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/timao/monitor-qmpmonitor1-20180220-094308-h9I6hRsI,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \ 2. Boot guest in target # /usr/libexec/qemu-kvm \ -name 'guest' \ -machine q35 \ -nodefaults \ -vga qxl \ -vnc :2 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x3 \ -blockdev driver=nbd,cache.direct=on,cache.no-flush=off,server.host=localhost,server.port=9000,server.type=inet,node-name=my_file \ -blockdev driver=raw,node-name=my,file=my_file \ -device scsi-hd,drive=my \ -monitor stdio \ -m 8192 \ -smp 8 \ -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9 \ -netdev tap,id=idxgXAlm \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/timao/monitor-qmpmonitor1-20180220-094308-h9I6hRsI,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \ -incoming tcp:0:5200 3. Migration and cancel in source (qemu) migrate -d tcp:0:5200 (qemu) migrate_cancel Result: Target: (qemu) qemu-kvm: check_section_footer: Read section footer failed: -5 qemu-kvm: load of migration failed: Invalid argument Server: # qemu-nbd -f qcow2 sn.qcow2 -t -p 9000 --share=2 Disconnect client, due to: Unexpected end-of-file before all bytes were read Disconnect client, due to: Unexpected end-of-file before all bytes were read Disconnect client, due to: Unexpected end-of-file before all bytes were read
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks
Reproduced this issue with : qemu-kvm-5.1.0-10.module+el8.3.0+8254+568ca30d kernel-4.18.0-234 Same steps as Comment9: qemu-nbd -f qcow2 /var/lib/libvirt/images/f31.qcow2 -p 9000 -t --share=2 1- Boot source guest /usr/libexec/qemu-kvm \ -name 'guest' \ -machine q35 \ -nodefaults \ -vga qxl \ -vnc :1 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x3 \ -blockdev driver=nbd,cache.direct=on,cache.no-flush=off,server.host=localhost,server.port=9000,server.type=inet,node-name=my_file \ -blockdev driver=raw,node-name=my,file=my_file \ -device scsi-hd,drive=my \ -monitor stdio \ -m 8192 \ -smp 8 \ -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9 \ -netdev tap,id=idxgXAlm \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 2- Boot dest guest /usr/libexec/qemu-kvm \ -name 'guest' \ -machine q35 \ -nodefaults \ -vga qxl \ -vnc :2 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x3 \ -blockdev driver=nbd,cache.direct=on,cache.no-flush=off,server.host=localhost,server.port=9000,server.type=inet,node-name=my_file \ -blockdev driver=raw,node-name=my,file=my_file \ -device scsi-hd,drive=my \ -monitor stdio \ -m 8192 \ -smp 8 \ -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b3,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pcie.0,addr=0x9 \ -netdev tap,id=idxgXAlm \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-target,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -device pcie-root-port,id=pcie.0-root-port-8,slot=8,chassis=8,addr=0x8,bus=pcie.0 \ -incoming tcp:0:5200 3- Migration and cancel in source (qemu) migrate -d tcp:0:5200 (qemu) migrate_cancel Result: In target: qemu-kvm: error while loading state section id 1(ram) qemu-kvm: load of migration failed: Invalid argument In server: qemu-nbd: Disconnect client, due to: Failed to read request: Unexpected end-of-file before all bytes were read qemu-nbd: Disconnect client, due to: Failed to read request: Unexpected end-of-file before all bytes were read qemu-nbd: Disconnect client, due to: Failed to read request: Unexpected end-of-file before all bytes were read
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
With qemu-kvm-5.2.0-1.module+el8.4.0+9091+650b220a.x86_64, it reproduce the bug, but libvirt doesn't support such option 'migrate_cancel' Hence, agree to closed this.