Bug 1493890

(In reply to Longxiang Lyu from comment #0)
> Description of problem:
> qemu hangs if shut down NBD service then make snapshot of the extra NBD disk.
> 
> Version-Release number of selected component (if applicable):
> kernel-3.10.0-709.el7.x86_64
> qemu-kvm-rhev-2.9.0-16.el7_4.8
> 
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. use qemu to  export a disk as NBD driver
> # qemu-kvm -drive file=test.qcow2,format=raw,id=img0 -qmp
> tcp:0:5555,server,nowait -monitor stdio -incoming tcp:0:6666
> qmp:
> { "execute": "qmp_capabilities" }
> { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet",
> "data": { "host": "10.66.11.1", "port": "9000" } } } }
> { "execute": "nbd-server-add", "arguments": { "device": "img0", "writable":
> true } }

Can you reproduce the VM hanging by using qemu-nbd instead of qemu-kvm as the NBD export?
qemu-nbd -f raw test.qcow2 -p 9000 -x img0

> 
> 2. boot up a VM with the NBD image as additional driver
> ...
> -drive
> file=/home/test/streamnbd/test.raw,format=raw,if=none,cache=none,
> snapshot=off,rerror=stop,werror=stop,id=img0 \
> -device ide-hd,bus=ide.0,unit=0,drive=img0,id=ide-disk0,bootindex=0 \
> -drive
> file=nbd://10.66.11.1:9000/img0,format=qcow2,if=none,cache=none,snapshot=off,
> rerror=stop,werror=stop,id=img1 \
> -device ide-hd,bus=ide.0,unit=1,drive=img1,id=ide-disk1 \
> …
> 

This requests that on EIO error on the NBD drive, the guest should stop...

> 3. stop nbd service
> { "execute" : "nbd-server-stop", "arguments" : {} }

...and this should cause an EIO error on the next access to the NBD drive

> 
> 4. do snapshot of img1
> { "execute": "blockdev-snapshot-sync", "arguments": { "device": "img1",
> "snapshot-file": "/home/test/streamnbd/sn1.qcow2", "format": "qcow2",
> "mode": "absolute-paths" } }
> 
> 
> Actual results:
> guest hangs.
> nothing returns after the snapshot command.

I'm suspecting that this is yet another symptom of NBD client fixes that went in just before 2.10 was released; I'll try to reproduce it and double check whether this is a duplicate.

> 
> Expected results:
> guest functions normally and qemu should output error of snapshot

The guest may be stopped at this point (due to the werror policy), but the QMP monitor should remain responsive.

The set of NBD fixes that made it into 2.10 is incomplete compared to what I tested for fixing the bug in 2.9; therefore, even RHEL 7.5 will need backports (I'm waiting for the 7.5 build tree to open up). I'm adding 7.4.z?, since we probably also want this in z-stream.

The following commits on top of 2.9.0-16.el7_4.8 are needed:
already in 2.10:
72b6ffc7
d0a18013
d3d7d03f
missed 2.10:
3c2db183
6e592fc9
02d2d860
a6934370

(In reply to Eric Blake from comment #2)
> (In reply to Longxiang Lyu from comment #0)
> > Description of problem:
> > qemu hangs if shut down NBD service then make snapshot of the extra NBD disk.
> > 
> > Version-Release number of selected component (if applicable):
> > kernel-3.10.0-709.el7.x86_64
> > qemu-kvm-rhev-2.9.0-16.el7_4.8
> > 
> > 
> > How reproducible:
> > 100%
> > 
> > Steps to Reproduce:
> > 1. use qemu to  export a disk as NBD driver
> > # qemu-kvm -drive file=test.qcow2,format=raw,id=img0 -qmp
> > tcp:0:5555,server,nowait -monitor stdio -incoming tcp:0:6666
> > qmp:
> > { "execute": "qmp_capabilities" }
> > { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet",
> > "data": { "host": "10.66.11.1", "port": "9000" } } } }
> > { "execute": "nbd-server-add", "arguments": { "device": "img0", "writable":
> > true } }
> 
> Can you reproduce the VM hanging by using qemu-nbd instead of qemu-kvm as
> the NBD export?
> qemu-nbd -f raw test.qcow2 -p 9000 -x img0
> 
> > 
> > 2. boot up a VM with the NBD image as additional driver
> > ...
> > -drive
> > file=/home/test/streamnbd/test.raw,format=raw,if=none,cache=none,
> > snapshot=off,rerror=stop,werror=stop,id=img0 \
> > -device ide-hd,bus=ide.0,unit=0,drive=img0,id=ide-disk0,bootindex=0 \
> > -drive
> > file=nbd://10.66.11.1:9000/img0,format=qcow2,if=none,cache=none,snapshot=off,
> > rerror=stop,werror=stop,id=img1 \
> > -device ide-hd,bus=ide.0,unit=1,drive=img1,id=ide-disk1 \
> > …
> > 
> 
> This requests that on EIO error on the NBD drive, the guest should stop...
> 
> > 3. stop nbd service
> > { "execute" : "nbd-server-stop", "arguments" : {} }
> 
> ...and this should cause an EIO error on the next access to the NBD drive
> 
> > 
> > 4. do snapshot of img1
> > { "execute": "blockdev-snapshot-sync", "arguments": { "device": "img1",
> > "snapshot-file": "/home/test/streamnbd/sn1.qcow2", "format": "qcow2",
> > "mode": "absolute-paths" } }
> > 
> > 
> > Actual results:
> > guest hangs.
> > nothing returns after the snapshot command.
> 
> I'm suspecting that this is yet another symptom of NBD client fixes that
> went in just before 2.10 was released; I'll try to reproduce it and double
> check whether this is a duplicate.
> 
> > 
> > Expected results:
> > guest functions normally and qemu should output error of snapshot
> 
> The guest may be stopped at this point (due to the werror policy), but the
> QMP monitor should remain responsive.


Yes, I could reproduce this with qemu-nbd.
1. export addition.qcow2
# qemu-nbd -f raw addition.qcow2 -p 9000 -t -x addition.qcow2

2. boot up a VM with addition.qcow2 as second disk
-drive file=/home/test/streamnbd/test.raw,format=raw,if=none,cache=none,snapshot=off,rerror=stop,werror=stop,id=img0 \
-device virtio-blk-pci,drive=img0,id=disk0,bootindex=0 \
-drive file=nbd://10.66.11.1:9000/addition.qcow2,format=qcow2,if=none,cache=none,snapshot=off,rerror=stop,werror=stop,id=img1 \
-device virtio-blk-pci,drive=img1,id=disk1 \

3. kill qemu-nbd
# kill -9 $(ps aux | grep qemu-nbd | head -n1 | awk '{ print $2 }')

4. do snapshot
{ "execute": "blockdev-snapshot-sync", "arguments": { "device": "img1", "snapshot-file": "/home/test/streamnbd/sn1.qcow2", "format": "qcow2", "mode": "absolute-paths" } }

result: qemu hangs

*** This bug has been marked as a duplicate of bug 1482478 ***