Bug 1262676

Summary: When mirroring to remote NBD disk with granularity =8192 and buf-size=8193, qemu core dump ( on src host)
Product: Red Hat Enterprise Linux 7 Reporter: Pei Zhang <pezhang>
Component: qemu-kvm-rhevAssignee: Hanna Czenczek <hreitz>
Status: CLOSED ERRATA QA Contact: Qianqian Zhu <qizhu>
Severity: high Docs Contact:
Priority: low    
Version: 7.2CC: chayang, famz, huding, juzhang, knoel, michen, mrezanin, pezhang, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 2.4.0 Doc Type: If docs needed, set a value
Doc Text:
Cause: drive-mirror internally expects buf-size to be a multiple of granularity. Consequence: If it was not, qemu would crash. Fix: buf-size will now be aligned automatically to granularity. Result: qemu no longer crashes when buf-size is not specified as a multiple of granularity.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 23:29:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pei Zhang 2015-09-14 05:27:52 UTC
Description of problem:
On src host, when doing block mirror to the remote NBD disk with granularity =8192 and buf-size=8193, qemu core dumped. 
If buf-size is power of 2, qemu works well. But if not, certain values(such as  8193,16385) will cause qemu core dumped.


Version-Release number of selected component (if applicable):
Host (src & des):
Kernel:3.10.0-315.el7.x86_64
qemu-kvm-rhev:qemu-kvm-rhev-2.3.0-22.el7.x86_64

Guest: Win10 32


How reproducible:
100%

Steps to Reproduce:
1. boot guest on src host
# /usr/libexec/qemu-kvm -name win10_32_src -machine pc-i440fx-rhel7.2.0,accel=kvm \
-cpu SandyBridge -m 4G,slots=256,maxmem=40G -numa node \
-smp 4,sockets=2,cores=2,threads=1 \
-uuid 82b1a01e-5f6c-4f5f-8d27-3855a74e6b6b \
-netdev tap,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=12:54:00:5c:88:6d \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
-spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \
-monitor stdio \
-serial unix:/tmp/monitor,server,nowait \
-qmp tcp:0:5555,server,nowait \
-drive file=/home/win10_32.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \ 

2. boot guest(-incoming) on des host with a empty disk
# qemu-img create -f qcow2 -o compat=0.10 /home/win10_32.qcow2 50G
# /usr/libexec/qemu-kvm -name win10_32_src -machine pc-i440fx-rhel7.2.0,accel=kvm \
-cpu SandyBridge -m 4G,slots=256,maxmem=40G -numa node \
-smp 4,sockets=2,cores=2,threads=1 \
-uuid 82b1a01e-5f6c-4f5f-8d27-3855a74e6b6b \
-netdev tap,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=12:54:00:5c:88:6d \
-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16 \
-spice port=5900,addr=0.0.0.0,disable-ticketing,image-compression=off,seamless-migration=on \
-monitor stdio \
-serial unix:/tmp/monitor,server,nowait \
-qmp tcp:0:5555,server,nowait \
-drive file=/home/win10_32.qcow2,format=qcow2,if=none,id=drive-virtio-blk0,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive-virtio-blk0,id=virtio-blk0 \
-incoming tcp:0:6666 \

3. on des host, create NBD server, and export the empty disk
{ "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "10.66.9.120", "port": "3333" } } } }
{ "execute": "nbd-server-add", "arguments": { "device": "drive-virtio-blk0","writable": true } }

4. on src host, start mirroring to the remote NBD disk in step 3, with granularity =8192 and buf-size=8193
{"execute":"qmp_capabilities"}
{ "execute": "drive-mirror", "arguments": { "device": "drive-virtio-blk0","target": "nbd://10.66.9.120:3333/drive-virtio-blk0", "sync": "full","format": "raw", "mode": "existing","granularity":8192,"buf-size":8193,"on-source-error":"report","on-target-error":"report" } }

5. on src host, qemu core dumped.
src:
(qemu) Segmentation fault (core dumped)

des:
(qemu) nbd.c:nbd_receive_request():L804: read failed


Actual results:
on src host, qemu core dumped.
(qemu) Segmentation fault (core dumped)


Expected results:
qemu works well, Or disallow user's command and promote error message, such as  'Parameter 'buf-size' expects power of 2' or something else.


Additional info:
"granularity":65536,"buf-size":8193     works well
"granularity":8192,"buf-size":10485760  works well
"granularity":8192,"buf-size":8193      fail

"granularity":8192,"buf-size":8191      works well
"granularity":8192,"buf-size":8192      works well
"granularity":8192,"buf-size":8973      fail
"granularity":8192,"buf-size":16384     works well
"granularity":8192,"buf-size":16385     fail

Comment 6 Hanna Czenczek 2017-01-16 17:04:16 UTC
Fixed in upstream commit 48ac0a4df84662f23da25262443e1810b70c2228.

Max

Comment 8 Qianqian Zhu 2017-03-09 05:43:23 UTC
Reproduced with:
qemu-kvm-rhev-10:2.3.0-22.el7
kernel-3.10.0-315.el7.x86_64

Steps:
1. Launch guest on both source and destination:
Source: /usr/libexec/qemu-kvm -name linux -cpu SandyBridge -m 2048 -drive file=/nfs/rhel74-64-virtio.qcow2,if=none,cache=writeback,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0  -qmp tcp:0:5555,server,nowait -monitor stdio
Destination: /usr/libexec/qemu-kvm -name linux -cpu SandyBridge -m 2048 -drive file=/home/test,if=none,cache=writeback,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0  -qmp tcp:0:5556,server,nowait -monitor stdio -incoming tcp:0:6666

2. On destiantion, create NBD server, and export disk
{ "execute": "nbd-server-start", "arguments": { "addr": { "type":"inet","data": { "host": "10.66.8.124", "port": "3333" } } } }
{ "execute": "nbd-server-add", "arguments": { "device": "drive-virtio-disk0","writable": true } }

3. Start mirroring from soruce to the remote NBD disk in step 3, with granularity =8192 and buf-size=8193
{ "execute": "drive-mirror", "arguments": { "device": "drive-virtio-disk0","target": "nbd://10.66.8.124:3333/drive-virtio-disk0", "sync":"full","format": "raw", "mode": "existing","granularity":8192,"buf-size":8193,"on-source-error":"report","on-target-error":"report" } }

Result:
qemu core dump on source and read failed on destination:
Source: (qemu) Segmentation fault (core dumped)
Destination: (qemu) nbd.c:nbd_receive_request():L804: read failed


Verified with:
qemu-kvm-rhev-2.8.0-5.el7.x86_64
kernel-3.10.0-514.el7.x86_64

Steps:
Same as above.

Result:
Block mirror success.
{"timestamp": {"seconds": 1489037737, "microseconds": 986235}, "event": "BLOCK_JOB_READY", "data": {"device": "drive-virtio-disk0", "len": 21475041280, "offset": 21475041280, "speed": 0, "type": "mirror"}}

Comment 10 errata-xmlrpc 2017-08-01 23:29:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 11 errata-xmlrpc 2017-08-02 01:07:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 12 errata-xmlrpc 2017-08-02 01:59:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 13 errata-xmlrpc 2017-08-02 02:40:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 14 errata-xmlrpc 2017-08-02 03:04:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Comment 15 errata-xmlrpc 2017-08-02 03:24:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392