Bug 1778593 - Qemu coredump when backup to a existing small size image
Summary: Qemu coredump when backup to a existing small size image
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.2
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: 8.2
Assignee: Kevin Wolf
QA Contact: aihua liang
URL:
Whiteboard:
: 1824628 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-02 06:21 UTC by aihua liang
Modified: 2020-07-28 07:13 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-4.2.0-24.module+el8.2.1+6959+9b840e7c
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-07-28 07:12:15 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:3172 0 None None None 2020-07-28 07:13:37 UTC

Description aihua liang 2019-12-02 06:21:35 UTC
Description of problem:
 Qemu coredump when backup to a existing small size image

Version-Release number of selected component (if applicable):
 kernel version:4.18.0-147.el8.x86_64
 qemu-kvm version:qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64

How reproducible:
 100%

Steps to Reproduce:
1.Create a small target image
   qemu-img create -f qcow2 full.img 2G
2.Start guest with qemu cmd:
   /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1 \
    -m 7168  \
    -smp 4,maxcpus=4,cores=2,threads=1,dies=1,sockets=2  \
    -cpu 'Skylake-Client',+kvm_pv_unhalt  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20191118-011823-gEG3j1ms,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20191118-011823-gEG3j1mt,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=id4p8G4l \
    -chardev socket,server,id=chardev_serial0,path=/var/tmp/serial-serial0-20191118-011823-gEG3j1mt,nowait \
    -device isa-serial,id=serial0,chardev=chardev_serial0  \
    -chardev socket,id=seabioslog_id_20191118-011823-gEG3j1mt,path=/var/tmp/seabios-20191118-011823-gEG3j1mt,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20191118-011823-gEG3j1mt,iobase=0x402 \
    -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
    -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
    -object iothread,id=iothread0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel820-64-virtio.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-3,addr=0x0,iothread=iothread0 \
    -drive id=drive_data1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/aliang/data.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
    -device virtio-blk-pci,id=data1,drive=drive_data1,bus=pcie.0-root-port-6,addr=0x0,iothread=iothread0 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-net-pci,mac=9a:4f:f4:e5:bd:67,id=idkQvhgf,netdev=idnMcj5J,bus=pcie.0-root-port-4,addr=0x0  \
    -netdev tap,id=idnMcj5J,vhost=on \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -device pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
    -monitor stdio \
    -qmp tcp:0:3000,server,nowait \
    -device virtio-serial-pci,id=virtio-serial0,bus=pcie.0,addr=0x8 \
    -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 \
    -device virtserialport,bus=virtio-serial0.0,chardev=qga0,id=qemu-ga0,name=org.qemu.guest_agent.0 \

3. Check system disk info online
   image: /home/kvm_autotest_root/images/rhel820-64-virtio.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 5.47 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

4. Do full backup to target image.
   { "execute": "drive-backup", "arguments": { "device": "drive_image1", "target": "full.img","sync": "full","format":"qcow2","mode":"existing"}}
   {"timestamp": {"seconds": 1575266125, "microseconds": 383650}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "drive_image1"}}
{"timestamp": {"seconds": 1575266125, "microseconds": 383707}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "drive_image1"}}
{"return": {}}

Actual results:
 After step4, wait some minutes, qemu coredump.
 (qemu) qemu-kvm: block/io.c:1871: bdrv_co_write_req_prepare: Assertion `end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE' failed.
test.txt: line 41: 25025 Aborted                 (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -machine q35 -nodefaults -device VGA,bus=pcie.0,addr=0x1 -m 7168 -smp 4,maxcpus=4,cores=2,threads=1,dies=1,sockets=2 -cpu 'Skylake-Client',+kvm_pv_unhalt ...

(gdb) bt
#0  0x00007f6ece3de8df in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f6ece3c8cf5 in __GI_abort () at abort.c:79
#2  0x00007f6ece3c8bc9 in __assert_fail_base
    (fmt=0x7f6ece52f300 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5578d6b109e8 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=0x5578d6b1045d "block/io.c", line=1871, function=<optimized out>) at assert.c:92
#3  0x00007f6ece3d6e96 in __GI___assert_fail
    (assertion=assertion@entry=0x5578d6b109e8 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=file@entry=0x5578d6b1045d "block/io.c", line=line@entry=1871, function=function@entry=0x5578d6b10ed0 <__PRETTY_FUNCTION__.26994> "bdrv_co_write_req_prepare") at assert.c:101
#4  0x00005578d68fe325 in bdrv_co_write_req_prepare
    (child=0x5578d8372750, child=0x5578d8372750, flags=2, req=0x7f6cd9ae3a50, bytes=65536, offset=2147483648) at block/io.c:1871
#5  0x00005578d68fe325 in bdrv_aligned_pwritev
    (child=child@entry=0x5578d8372750, req=req@entry=0x7f6cd9ae3a50, offset=offset@entry=2147483648, bytes=bytes@entry=65536, align=align@entry=1, qiov=qiov@entry=0x0, qiov_offset=0, flags=2)
    at block/io.c:1959
#6  0x00005578d68fe7ba in bdrv_co_do_zero_pwritev
    (req=0x7f6cd9ae3a50, flags=BDRV_REQ_ZERO_WRITE, bytes=65536, offset=2147483648, child=0x5578d8372750) at block/io.c:2055
#7  0x00005578d68fe7ba in bdrv_co_pwritev_part
    (child=child@entry=0x5578d8372750, offset=offset@entry=2147483648, bytes=bytes@entry=65536, qiov=qiov@entry=0x0, qiov_offset=qiov_offset@entry=0, flags=BDRV_REQ_ZERO_WRITE) at block/io.c:2127
#8  0x00005578d68fedda in bdrv_co_pwritev
    (flags=<optimized out>, qiov=0x0, bytes=65536, offset=2147483648, child=0x5578d8372750)
    at block/io.c:2158
--Type <RET> for more, q to quit, c to continue without paging--
#9  0x00005578d68fedda in bdrv_co_pwrite_zeroes
    (child=child@entry=0x5578d8372750, offset=offset@entry=2147483648, bytes=bytes@entry=65536, flags=<optimized out>, flags@entry=BDRV_REQ_ZERO_WRITE) at block/io.c:2158
#10 0x00005578d68ff25d in bdrv_co_copy_range_internal
    (src=src@entry=0x0, src_offset=0, dst=dst@entry=0x5578d8372750, dst_offset=dst_offset@entry=2147483648, bytes=65536, read_flags=read_flags@entry=0, write_flags=BDRV_REQ_ZERO_WRITE, recurse_src=true)
    at block/io.c:3201
#11 0x00005578d68ff745 in bdrv_co_copy_range_from
    (src=src@entry=0x0, src_offset=<optimized out>, dst=dst@entry=0x5578d8372750, dst_offset=dst_offset@entry=2147483648, bytes=<optimized out>, read_flags=read_flags@entry=0, write_flags=BDRV_REQ_ZERO_WRITE)
    at block/io.c:3270
#12 0x00005578d68c9098 in qcow2_co_copy_range_from
    (bs=0x5578d8385370, src=<optimized out>, src_offset=2147483648, dst=0x5578d8372750, dst_offset=2147483648, bytes=65536, read_flags=0, write_flags=0) at block/qcow2.c:3846
#13 0x00005578d68ff0cf in bdrv_co_copy_range_internal
    (src=0x5578d9758800, src_offset=src_offset@entry=2147483648, dst=0x5578d8372750, dst_offset=dst_offset@entry=2147483648, bytes=bytes@entry=65536, read_flags=read_flags@entry=0, write_flags=0, recurse_src=true) at block/io.c:3229
#14 0x00005578d68ff745 in bdrv_co_copy_range_from
    (src=<optimized out>, src_offset=src_offset@entry=2147483648, dst=<optimized out>, dst_offset=dst_offset@entry=2147483648, bytes=bytes@entry=65536, read_flags=read_flags@entry=0, write_flags=0)
    at block/io.c:3270
#15 0x00005578d68ff776 in bdrv_co_copy_range
    (src=<optimized out>, src_offset=src_offset@entry=2147483648, dst=<optimized out>, dst_offset=dst_offset@entry=2147483648, bytes=bytes@entry=65536, read_flags=read_flags@entry=0, write_flags=0)
    at block/io.c:3295
--Type <RET> for more, q to quit, c to continue without paging--
#16 0x00005578d690b295 in block_copy_do_copy
    (error_is_read=0x7f6cd9ae3f60, end=2147549184, start=2147483648, s=0x5578d85dac00)
    at block/block-copy.c:164
#17 0x00005578d690b295 in block_copy
    (s=0x5578d85dac00, start=start@entry=2147483648, bytes=bytes@entry=65536, error_is_read=error_is_read@entry=0x7f6cd9ae3f60) at block/block-copy.c:336
#18 0x00005578d6909d3f in backup_do_cow
    (error_is_read=0x7f6cd9ae3f60, bytes=65536, offset=2147483648, job=0x5578d91cafe0)
    at block/backup.c:83
#19 0x00005578d6909d3f in backup_loop (job=0x5578d91cafe0) at block/backup.c:205
#20 0x00005578d6909d3f in backup_run (job=0x5578d91cafe0, errp=<optimized out>) at block/backup.c:284
#21 0x00005578d68bb4d3 in job_co_entry (opaque=0x5578d91cafe0) at job.c:878
#22 0x00005578d6994983 in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>)
    at util/coroutine-ucontext.c:115
#23 0x00007f6ece3f44b0 in __start_context () at ../sysdeps/unix/sysv/linux/x86_64/__start_context.S:91
#24 0x00007f6ec6c24c00 in  ()
#25 0x0000000000000000 in  ()


Expected results:
  When backup to the target small image, should report block job "No space left" error.

Additional info:
  Will attach the coredump info later.

Comment 3 aihua liang 2019-12-03 05:59:44 UTC
Test on qemu-kvm-4.1.0-16.module+el8.1.1+4917+752cfd65.x86_64, don't hit this issue, it will backup failed with event:
 {"timestamp": {"seconds": 1575352695, "microseconds": 999482}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 21474836480, "offset": 2147483648, "speed": 0, "type": "backup", "error": "Input/output error"}}

Set this bug to a regression.

Comment 4 Ademar Reis 2019-12-04 01:06:11 UTC
(In reply to aihua liang from comment #0)
> Description of problem:
>  Qemu coredump when backup to a existing small size image
> 
> Version-Release number of selected component (if applicable):
>  kernel version:4.18.0-147.el8.x86_64
>  qemu-kvm version:qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64
> 
> How reproducible:
>  100%
> 
> Steps to Reproduce:
> 1.Create a small target image
>    qemu-img create -f qcow2 full.img 2G
> 2.Start guest with qemu cmd:
>    /usr/libexec/qemu-kvm \
>     -name 'avocado-vt-vm1' \
>     -machine q35  \
>     -nodefaults \
>     -device VGA,bus=pcie.0,addr=0x1 \
>     -m 7168  \
>     -smp 4,maxcpus=4,cores=2,threads=1,dies=1,sockets=2  \
>     -cpu 'Skylake-Client',+kvm_pv_unhalt  \
>     -chardev
> socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20191118-
> 011823-gEG3j1ms,server,nowait \
>     -mon chardev=qmp_id_qmpmonitor1,mode=control  \
>     -chardev
> socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20191118-
> 011823-gEG3j1mt,server,nowait \
>     -mon chardev=qmp_id_catch_monitor,mode=control \
>     -device pvpanic,ioport=0x505,id=id4p8G4l \
>     -chardev
> socket,server,id=chardev_serial0,path=/var/tmp/serial-serial0-20191118-
> 011823-gEG3j1mt,nowait \
>     -device isa-serial,id=serial0,chardev=chardev_serial0  \
>     -chardev
> socket,id=seabioslog_id_20191118-011823-gEG3j1mt,path=/var/tmp/seabios-
> 20191118-011823-gEG3j1mt,server,nowait \
>     -device
> isa-debugcon,chardev=seabioslog_id_20191118-011823-gEG3j1mt,iobase=0x402 \
>     -device
> pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
>     -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
>     -object iothread,id=iothread0 \
>     -drive
> id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,
> file=/home/kvm_autotest_root/images/rhel820-64-virtio.qcow2 \
>     -device
> pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
>     -device
> virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-
> 3,addr=0x0,iothread=iothread0 \
>     -drive
> id=drive_data1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,
> file=/home/aliang/data.qcow2 \
>     -device
> pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
>     -device
> virtio-blk-pci,id=data1,drive=drive_data1,bus=pcie.0-root-port-6,addr=0x0,
> iothread=iothread0 \
>     -device
> pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
>     -device
> virtio-net-pci,mac=9a:4f:f4:e5:bd:67,id=idkQvhgf,netdev=idnMcj5J,bus=pcie.0-
> root-port-4,addr=0x0  \
>     -netdev tap,id=idnMcj5J,vhost=on \
>     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
>     -vnc :0  \
>     -rtc base=utc,clock=host,driftfix=slew  \
>     -boot order=cdn,once=c,menu=off,strict=off \
>     -enable-kvm \
>     -device
> pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.
> 0 \
>     -monitor stdio \
>     -qmp tcp:0:3000,server,nowait \
>     -device virtio-serial-pci,id=virtio-serial0,bus=pcie.0,addr=0x8 \
>     -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 \
>     -device
> virtserialport,bus=virtio-serial0.0,chardev=qga0,id=qemu-ga0,name=org.qemu.
> guest_agent.0 \


Can you please retest with -blockdev?

Comment 5 aihua liang 2019-12-04 06:17:18 UTC
Test with -blockdev, also hit this issue.

1.Start guest with qemu cmds with system disk 20G.
    ...
    -blockdev driver=file,node-name=file_node,filename=/home/kvm_autotest_root/iamges/rhel820-64-virtio.qcow2 \
    -blockdev driver=qcow2,file=file_node,node-name=drive_image1 \
    -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \ 
    -device virtio-blk-pci,id=image1,drive=drive_data1,bus=pcie.0-root-port-6,addr=0x0 \
    ...

2.Create full backup target with size 2G.
   {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn1','size':2147483648},'job-id':'job1'}}"        
   {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn1','filename':'/root/sn1'}}
    {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn1','size':2147483648},'job-id':'job2'}}
    {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn$i','file':'drive_sn1'}}
    {'execute':'job-dismiss','arguments':{'id':'job1'}}
    {'execute':'job-dismiss','arguments':{'id':'job2'}}

3.Do full backup to small image and check block job info:
   { "execute": "transaction", "arguments": { "actions": [ {"type": "blockdev-backup", "data": { "device": "drive_image1", "target": "sn1", "sync": "full", "job-id":"j1" } }]}}
{"timestamp": {"seconds": 1575439863, "microseconds": 939287}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "j1"}}
{"timestamp": {"seconds": 1575439863, "microseconds": 939327}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "j1"}}
{"return": {}}

{"execute":"query-block-jobs"}
{"return": [{"auto-finalize": true, "io-status": "ok", "device": "j1", "auto-dismiss": true, "busy": true, "len": 21474836480, "offset": 536870912, "status": "running", "paused": false, "speed": 0, "ready": false, "type": "backup"}]}

Test Result:
  After step3, qemu core dump with info:
   qemu-kvm: block/io.c:1871: bdrv_co_write_req_prepare: Assertion `end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE' failed.
blockdev.txt: line 43:  6040 Aborted                 (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -machine q35 ...

  gdb info:
  #0  0x00007f65ec9d28df in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f65ec9bccf5 in __GI_abort () at abort.c:79
#2  0x00007f65ec9bcbc9 in __assert_fail_base
    (fmt=0x7f65ecb23300 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5594e2b3a9e8 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=0x5594e2b3a45d "block/io.c", line=1871, function=<optimized out>) at assert.c:92
#3  0x00007f65ec9cae96 in __GI___assert_fail
    (assertion=assertion@entry=0x5594e2b3a9e8 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=file@entry=0x5594e2b3a45d "block/io.c", line=line@entry=1871, function=function@entry=0x5594e2b3aed0 <__PRETTY_FUNCTION__.26994> "bdrv_co_write_req_prepare") at assert.c:101
#4  0x00005594e2928325 in bdrv_co_write_req_prepare
    (child=0x5594e5ccc5b0, child=0x5594e5ccc5b0, flags=0, req=0x7f63fc9e8a10, bytes=65536, offset=3223388160) at block/io.c:1871
#5  0x00005594e2928325 in bdrv_aligned_pwritev
    (child=child@entry=0x5594e5ccc5b0, req=req@entry=0x7f63fc9e8a10, offset=offset@entry=3223388160, bytes=65536, align=1, qiov=0x7f63fc9e8bc0, qiov_offset=0, flags=0)
    at block/io.c:1959
#6  0x00005594e292846b in bdrv_co_pwritev_part
    (child=0x5594e5ccc5b0, offset=<optimized out>, 
--Type <RET> for more, q to quit, c to continue without paging--
    offset@entry=3223388160, bytes=<optimized out>, 
    bytes@entry=65536, qiov=<optimized out>, 
    qiov@entry=0x7f63fc9e8bc0, qiov_offset=<optimized out>, 
    qiov_offset@entry=0, flags=<optimized out>) at block/io.c:2137
#7  0x00005594e292888f in bdrv_co_pwritev
    (child=<optimized out>, offset=offset@entry=3223388160, bytes=bytes@entry=65536, qiov=qiov@entry=0x7f63fc9e8bc0, flags=<optimized out>) at block/io.c:2087
#8  0x00005594e2935425 in bdrv_co_pwrite
    (flags=<optimized out>, buf=0x7f65d803a000, bytes=65536, offset=3223388160, child=<optimized out>)
    at /usr/src/debug/qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64/include/block/block_int.h:995
#9  0x00005594e2935425 in block_copy_do_copy
    (error_is_read=0x0, end=3223453696, start=3223388160, s=0x5594e49a3b60)
    at block/block-copy.c:193
#10 0x00005594e2935425 in block_copy
    (s=0x5594e49a3b60, start=3223388160, bytes=<optimized out>, error_is_read=error_is_read@entry=0x0) at block/block-copy.c:336
#11 0x00005594e293692b in backup_top_cbw
    (offset=offset@entry=3223388160, bytes=bytes@entry=16384, bs=<optimized out>)
    at block/backup-top.c:57
--Type <RET> for more, q to quit, c to continue without paging--
#12 0x00005594e293695e in backup_top_co_pwritev
    (bs=0x5594e4b54180, offset=3223388160, bytes=16384, qiov=0x7f65d800dbf8, flags=0)
    at block/backup-top.c:88
#13 0x00005594e2925adb in bdrv_driver_pwritev
    (bs=bs@entry=0x5594e4b54180, offset=offset@entry=3223388160, bytes=bytes@entry=16384, qiov=qiov@entry=0x7f65d800dbf8, qiov_offset=qiov_offset@entry=0, flags=flags@entry=0) at block/io.c:1183
#14 0x00005594e2927c7a in bdrv_aligned_pwritev
    (child=child@entry=0x5594e48f6dd0, req=req@entry=0x7f63fc9e8e20, offset=offset@entry=3223388160, bytes=16384, align=<optimized out>, qiov=0x7f65d800dbf8, qiov_offset=0, flags=0) at block/io.c:1980
#15 0x00005594e292846b in bdrv_co_pwritev_part
    (child=0x5594e48f6dd0, offset=<optimized out>, 
    offset@entry=3223388160, bytes=<optimized out>, 
    bytes@entry=16384, qiov=<optimized out>, 
    qiov@entry=0x7f65d800dbf8, qiov_offset=<optimized out>, 
    qiov_offset@entry=0, flags=flags@entry=0) at block/io.c:2137
#16 0x00005594e2915036 in blk_co_pwritev_part
    (blk=0x5594e5a17920, offset=3223388160, bytes=16384, qiov=qiov@entry=0x7f65d800dbf8, qiov_offset=qiov_offset@entry=0, flags=0) at block/block-backend.c:1211
#17 0x00005594e29150e7 in blk_co_pwritev
--Type <RET> for more, q to quit, c to continue without paging--
    (flags=<optimized out>, qiov=0x7f65d800dbf8, bytes=<optimized out>, offset=<optimized out>, blk=<optimized out>) at block/block-backend.c:1415
#18 0x00005594e29150e7 in blk_aio_write_entry (opaque=0x7f65d8008940)
    at block/block-backend.c:1415
#19 0x00005594e29be983 in coroutine_trampoline
    (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:115
#20 0x00007f65ec9e84b0 in __start_context ()
    at ../sysdeps/unix/sysv/linux/x86_64/__start_context.S:91
#21 0x00007f65e5218c00 in  ()
#22 0x0000000000000000 in  ()

Comment 7 John Snow 2019-12-13 23:19:38 UTC
This might be related to the new backup filters that got added in 4.2. I will investigate, thank you

Comment 8 Ademar Reis 2020-02-05 23:09:14 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 9 Peter Krempa 2020-04-17 06:51:49 UTC
*** Bug 1824628 has been marked as a duplicate of this bug. ***

Comment 10 Kevin Wolf 2020-04-29 11:35:49 UTC
Posted a fix upstream to make it fail gracefully and immediately when trying to start the job rather than only some time later:

https://lists.gnu.org/archive/html/qemu-block/2020-04/msg01355.html

Comment 12 Nir Soffer 2020-05-27 13:21:05 UTC
I understand that this is an issue when using backup push mode, so RHV
does not depend on this.

Comment 14 aihua liang 2020-06-02 10:36:24 UTC
Also hit the same issue when dd file on data disk that created with qcow2 but start with raw on backend:nfsv4.

Test Env:
  kernel version: 4.18.0-203.el8.x86_64
  qemu-kvm version: qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420
  #mount 10.66.8.210:/mnt/nfs /mnt/nfs


Test Steps:
  1. Guest start with:
    -blockdev node-name=file_data1,driver=file,aio=threads,filename=/mnt/nfs/data.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_data1,driver=raw,cache.direct=on,cache.no-flush=off,file=file_data1 \
    -device scsi-hd,id=data1,drive=drive_data1,write-cache=on \

  2. DD on data disk.
     #dd if=/dev/urandom of=/dev/sdb bs=1M count=1000
       no space left on device.

After step2, qemu crash with info:
  (qemu) qemu-kvm: /builddir/build/BUILD/qemu-5.0.0/block/io.c:1887: bdrv_co_write_req_prepare: Assertion `end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE' failed.
bug.txt: line 42: 146526 Aborted                 (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -sandbox on -machine q35 -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 -nodefaults -device VGA,bus=pcie.0,addr=0x2 -m 30720 ...

Gdb info:
  (gdb) bt
#0  0x00007f497641b7ff in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f4976405c35 in __GI_abort () at abort.c:79
#2  0x00007f4976405b09 in __assert_fail_base
    (fmt=0x7f497656bef0 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x557f10fbbf28 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=0x557f10fbbc48 "/builddir/build/BUILD/qemu-5.0.0/block/io.c", line=1887, function=<optimized out>) at assert.c:92
#3  0x00007f4976413de6 in __GI___assert_fail
    (assertion=assertion@entry=0x557f10fbbf28 "end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE", file=file@entry=0x557f10fbbc48 "/builddir/build/BUILD/qemu-5.0.0/block/io.c", line=line@entry=1887, function=function@entry=0x557f10fbc6f0 <__PRETTY_FUNCTION__.27275> "bdrv_co_write_req_prepare") at assert.c:101
#4  0x0000557f10da2ff7 in bdrv_co_write_req_prepare
    (child=0x557f11b3a4e0, child=0x557f11b3a4e0, flags=0, req=0x7f419aaf1ab0, bytes=73728, offset=126976)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:1887
#5  0x0000557f10da2ff7 in bdrv_aligned_pwritev
    (child=child@entry=0x557f11b3a4e0, req=req@entry=0x7f419aaf1ab0, offset=126976, bytes=73728, align=4096, qiov=0x7f419aaf1b70, qiov_offset=0, flags=0) at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:1975
#6  0x0000557f10da314b in bdrv_co_pwritev_part (child=0x557f11b3a4e0, offset=<optimized out>, 
    offset@entry=130048, bytes=<optimized out>, bytes@entry=67072, qiov=<optimized out>, 
    qiov@entry=0x557f11e52430, qiov_offset=<optimized out>, qiov_offset@entry=0, flags=<optimized out>)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:2163
#7  0x0000557f10da352f in bdrv_co_pwritev
    (child=<optimized out>, offset=offset@entry=130048, bytes=bytes@entry=67072, qiov=qiov@entry=0x557f11e52430, flags=<optimized out>) at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:2102
#8  0x0000557f10d64e52 in raw_co_pwritev
    (bs=0x557f11b546e0, offset=130048, bytes=67072, qiov=0x557f11e52430, flags=<optimized out>)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/raw-format.c:258
#9  0x0000557f10da08bb in bdrv_driver_pwritev
    (bs=bs@entry=0x557f11b546e0, offset=offset@entry=130048, bytes=bytes@entry=67072, qiov=qiov@entry=0x557f11e52430, qiov_offset=qiov_offset@entry=0, flags=flags@entry=0)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:1187
#10 0x0000557f10da297c in bdrv_aligned_pwritev
    (child=child@entry=0x557f11b1baa0, req=req@entry=0x7f419aaf1e30, offset=130048, bytes=67072, align=<optimized out>, qiov=0x--Type <RET> for more, q to quit, c to continue without paging--
557f11e52430, qiov_offset=0, flags=0) at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:1996
#11 0x0000557f10da314b in bdrv_co_pwritev_part (child=0x557f11b1baa0, offset=<optimized out>, 
    offset@entry=130048, bytes=<optimized out>, bytes@entry=67072, qiov=<optimized out>, 
    qiov@entry=0x557f11e52430, qiov_offset=<optimized out>, qiov_offset@entry=0, flags=flags@entry=0)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/io.c:2163
#12 0x0000557f10d8fbf8 in blk_do_pwritev_part
    (blk=0x557f12f7b820, offset=130048, bytes=67072, qiov=0x557f11e52430, qiov_offset=qiov_offset@entry=0, flags=0)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/block-backend.c:1231
#13 0x0000557f10d8fcd1 in blk_aio_write_entry (opaque=0x557f12177380)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/block/block-backend.c:1439
#14 0x0000557f10e3cd33 in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>)
    at /usr/src/debug/qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64/util/coroutine-ucontext.c:115
#15 0x00007f49764313d0 in __start_context () at ../sysdeps/unix/sysv/linux/x86_64/__start_context.S:91
#16 0x00007ffc80a078a0 in  ()
#17 0x0000000000000000 in  ()

 Test the same scenario with backend:localfs, don't hit this issue.

Comment 19 aihua liang 2020-06-09 08:27:23 UTC
Test on qemu-kvm-4.2.0-24.module+el8.2.1+6959+9b840e7c, the problem has been resolved, so set bug's status to "Verified".


Test Env:
  kernel version:4.18.0-193.el8.x86_64
  qemu-kvm version: qemu-kvm-4.2.0-24.module+el8.2.1+6959+9b840e7c

Test Steps:
  1. Start guest with qemu cmds:
     -object iothread,id=iothread0 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/rhel821-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,write-cache=on,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \

  2. Create a small target.
      {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn1','size':524288000},'job-id':'job1'}}
      {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn1','filename':'/root/sn1'}}
      {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn1','size':524288000},'job-id':'job2'}}
      {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn1','file':'drive_sn1'}}
      {'execute':'job-dismiss','arguments':{'id':'job1'}}
      {'execute':'job-dismiss','arguments':{'id':'job2'}}

  3. Do full backup
      { "execute": "transaction", "arguments": { "actions": [ {"type": "blockdev-backup", "data": { "device": "drive_image1", "target": "sn1", "sync": "full", "job-id":"j1" } }, {"type": "block-dirty-bitmap-add", "data": { "node": "drive_image1", "name": "bitmap0" } } ] } }
{"error": {"class": "GenericError", "desc": "Source and target image have different sizes"}}


 After step3, test failed with error info ""Source and target image have different sizes".

Comment 21 errata-xmlrpc 2020-07-28 07:12:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3172


Note You need to log in before you can comment on or make changes to this bug.