Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1120042

Summary: Segmentation fault at block/qcow2-cluster.c:462
Product: Red Hat Enterprise Linux 6 Reporter: Xu Han <xuhan>
Component: qemu-kvmAssignee: Jeff Cody <jcody>
Status: CLOSED WORKSFORME QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.6CC: bsarathy, chayang, coli, juzhang, mkenneth, qzhang, rbalakri, scui, shu, virt-maint, xuhan
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-02 02:42:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
gdb bt full none

Description Xu Han 2014-07-16 06:40:12 UTC
Created attachment 918333 [details]
gdb bt full

Description of problem:
QEMU met segmentation fault during block streaming.
-----------------------------------------------------------------------------
Core was generated by `/usr/bin/qemu-kvm -S -name virt-tests-vm1 -M rhel6.5.0 -nodefaults -vga qxl -gl'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f3b9b3dc2a9 in qcow2_get_cluster_offset (bs=0x7f3b9c71d010, offset=14505934848, num=0x7f3b9f96e73c, cluster_offset=0x7f3b9f96e730)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2-cluster.c:462
462	    if (!l2_offset) {
-----------------------------------------------------------------------------
(gdb) bt
#0  0x00007f3b9b3dc2a9 in qcow2_get_cluster_offset (bs=0x7f3b9c71d010, offset=14505934848, num=0x7f3b9f96e73c, cluster_offset=0x7f3b9f96e730)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2-cluster.c:462
#1  0x00007f3b9b3d7fe1 in qcow2_co_readv (bs=0x7f3b9c71d010, sector_num=<value optimized out>, remaining_sectors=64, qiov=0x7f3b9f96e8b0)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2.c:544
#2  0x00007f3b9b3bb59f in bdrv_co_do_readv (bs=0x7f3b9c71d010, sector_num=28331904, nb_sectors=64, qiov=0x7f3b9f96e8b0, flags=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:2205
#3  0x00007f3b9b3bbbce in bdrv_rw_co_entry (opaque=0x7f3b9f96e880) at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:1885
#4  0x00007f3b9b3bdad8 in bdrv_rw_co (bs=0x7f3b9c71d010, sector_num=<value optimized out>, buf=<value optimized out>, nb_sectors=<value optimized out>, 
    is_write=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:1929
#5  0x00007f3b9b3dd737 in qcow2_read (bs=0x7f3b9c3b1010, start_sect=<value optimized out>, cluster_offset=9894494208, n_start=0, n_end=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2-cluster.c:339
#6  copy_sectors (bs=0x7f3b9c3b1010, start_sect=<value optimized out>, cluster_offset=9894494208, n_start=0, n_end=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2-cluster.c:382
#7  0x00007f3b9b3dd930 in qcow2_alloc_cluster_link_l2 (bs=0x7f3b9c3b1010, m=0x7f3b9f96eab0) at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2-cluster.c:663
#8  0x00007f3b9b3d794f in qcow2_co_writev (bs=0x7f3b9c3b1010, sector_num=<value optimized out>, remaining_sectors=520, qiov=0x7f3b9cba5f08)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block/qcow2.c:753
#9  0x00007f3b9b3bba53 in bdrv_co_do_writev (bs=0x7f3b9c3b1010, sector_num=28331968, nb_sectors=520, qiov=0x7f3b9cba5f08, flags=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:2321
#10 0x00007f3b9b3bbb11 in bdrv_co_do_rw (opaque=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/block.c:4057
#11 0x00007f3b9b3c52eb in coroutine_trampoline (i0=<value optimized out>, i1=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/coroutine-ucontext.c:129
#12 0x00007f3b97e39bf0 in ?? () from /lib64/libc.so.6
#13 0x00007fff44bcb670 in ?? ()
#14 0x0000000000000000 in ?? ()

Version-Release number of selected component (if applicable):
qemu-img-rhev-0.12.1.2-2.429.el6.x86_64

How reproducible:
Sometimes

Steps to Reproduce:
1. Create a snaphot.
{"execute": "blockdev-snapshot-sync", "arguments": {"device": "drive_image1", "snapshot-file": "/usr/local/staf/test/RHEV/kvm/autotest-devel/client/tests/virt/shared/data/images/sn1", "format": "qcow2"}, "id": "hcXTpZt0"}

2. Do streaming.
{"execute": "block-stream", "arguments": {"device": "drive_image1", "speed": 1048576}, "id": "BPtZff13"}

Actual results:
Core dumped.

Expected results:
The block job should finish with no error.

Additional info:
1. Last QMP logs:
----------------
2014-07-08 00:25:05: {"execute": "query-block-jobs", "id": "Y6oPvlbL"}
2014-07-08 00:25:05: {"return": [{"device": "drive_image1", "len": 32212254720, "offset": 25239748608, "speed": 10485760, "type": "stream"}], "id": "Y6oPvlbL"}
2014-07-08 00:25:06: {"execute": "query-status", "id": "fJEPsORL"}
2014-07-08 00:25:06: {"return": {"status": "running", "singlestep": false, "running": true}, "id": "fJEPsORL"}
2014-07-08 00:25:06: {"execute": "screendump", "arguments": {"filename": "/dev/shm/scrdump-LH5dhX.ppm"}, "id": "duA43BM9"}
2014-07-08 00:25:06: {"return": {}, "id": "duA43BM9"}
2014-07-08 00:25:06: {"execute": "query-block-jobs", "id": "EYC48Yyo"}
2014-07-08 00:25:06: {"return": [{"device": "drive_image1", "len": 32212254720, "offset": 31458263040, "speed": 10485760, "type": "stream"}], "id": "EYC48Yyo"}
2014-07-08 00:25:07: {"execute": "query-block-jobs", "id": "FzL4NnTU"}

2. QEMU command line:
------------------
/usr/bin/qemu-kvm \
    -S  \
    -name 'virt-tests-vm1' \
    -M rhel6.5.0  \
    -nodefaults  \
    -vga qxl  \
    -global qxl-vga.vram_size=33554432 \
    -device AC97,bus=pci.0,addr=03  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20140707-235147-2xAI2d2T,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140707-235147-2xAI2d2T,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20140707-235147-2xAI2d2T,path=/tmp/seabios-20140707-235147-2xAI2d2T,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140707-235147-2xAI2d2T,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=04 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=05 \
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,file=/usr/local/staf/test/RHEV/kvm/autotest-devel/client/tests/virt/shared/data/images/win7-64-sp1-virtio.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1 \
    -device virtio-net-pci,mac=9a:67:68:69:6a:6b,id=idTHP0SR,vectors=4,netdev=idRVWPZu,bus=pci.0,addr=06  \
    -netdev tap,id=idRVWPZu,vhost=on,vhostfd=28,fd=27  \
    -m 4096  \
    -smp 2,maxcpus=2,cores=1,threads=1,sockets=2  \
    -cpu 'Opteron_G3',hv_relaxed \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/usr/local/staf/test/RHEV/kvm/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso \
    -device scsi-cd,id=cd1,drive=drive_cd1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -spice port=3000,password=123456,addr=0,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off \
    -enable-kvm

Comment 1 Xu Han 2014-07-16 06:46:22 UTC
(gdb) p *s
$1 = {
  cluster_bits = 16, 
  cluster_size = 65536, 
  cluster_sectors = 128, 
  l2_bits = 13, 
  l2_size = 8192, 
  l1_size = 62, 
  l1_vm_state_index = 60, 
  csize_shift = 54, 
  csize_mask = 255, 
  cluster_offset_mask = 18014398509481983, 
  l1_table_offset = 196608, 
  l1_table = 0x0, 
  l2_table_cache = 0x7f3b9c3b1f20, 
  refcount_block_cache = 0x7f3b9c3aefc0, 
  cluster_cache = 0x7f3b9c4f5610 "", 
  cluster_data = 0x7f3b9153d010 "", 
  cluster_cache_offset = 18446744073709551615, 
  cluster_allocs = {
    lh_first = 0x0
  }, 
  refcount_table = 0x7f3b9c505620, 
  refcount_table_offset = 65536, 
  refcount_table_size = 8192, 
  free_cluster_index = 0, 
  free_byte_offset = 0, 
  lock = {
    locked = true, 
    queue = {
      entries = {
        tqh_first = 0x0, 
        tqh_last = 0x7f3b9c3b1a90
      }
    }
  }, 
  crypt_method = 0, 
  crypt_method_header = 0, 
  aes_encrypt_key = {
    rd_key = {0 <repeats 60 times>}, 
    rounds = 0
  }, 
  aes_decrypt_key = {
    rd_key = {0 <repeats 60 times>}, 
    rounds = 0
  }, 
  snapshots_offset = 0, 
  snapshots_size = 0, 
  nb_snapshots = 0, 
  snapshots = 0x0, 
  overlap_check = 127, 
  unknown_header_ext = {
    lh_first = 0x0
  }
}

Comment 4 juzhang 2014-07-16 23:24:43 UTC
Hi Xu,

Does rhel7.1 host hit this issue as well?

Best Regards,
Junyi

Comment 5 Xu Han 2014-07-17 10:55:32 UTC
(In reply to juzhang from comment #4)
> Hi Xu,
> 
> Does rhel7.1 host hit this issue as well?
> 
> Best Regards,
> Junyi

Tested 10 times block streaming on rhel7.1 host, did not hit this issue.

virt.qemu.smp_2.4096m.repeat1.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1763     completed successfully
virt.qemu.smp_2.4096m.repeat2.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1674     completed successfully
virt.qemu.smp_2.4096m.repeat3.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1537     completed successfully
virt.qemu.smp_2.4096m.repeat4.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1553     completed successfully
virt.qemu.smp_2.4096m.repeat5.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1575     completed successfully
virt.qemu.smp_2.4096m.repeat6.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1516     completed successfully
virt.qemu.smp_2.4096m.repeat7.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1513     completed successfully
virt.qemu.smp_2.4096m.repeat8.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1558     completed successfully
virt.qemu.smp_2.4096m.repeat9.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                             GOOD       1907     completed successfully
virt.qemu.smp_2.4096m.repeat10.run_test.Host_RHEL.7.1.qcow2.virtio_scsi.up.virtio_net.Win7.x86_64.sp1.io-github-autotest-qemu.block_stream.simple_test.reset_speed                            GOOD       1576     completed successfully