RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1083413 - qemu-kvm: iSCSI: Failure. SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
Summary: qemu-kvm: iSCSI: Failure. SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.0
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: rc
: ---
Assignee: Fam Zheng
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 1025342 (view as bug list)
Depends On:
Blocks: RHEL7.0Virt-PostBeta(0-Day) 1090978
TreeView+ depends on / blocked
 
Reported: 2014-04-02 08:17 UTC by Sibiao Luo
Modified: 2015-03-05 08:05 UTC (History)
14 users (show)

Fixed In Version: qemu-kvm-1.5.3-61.el7
Doc Type: Bug Fix
Doc Text:
Due to a bug in the Internet Small Computer System Interface (iSCSI) driver, a qemu-kvm process terminated unexpectedly with a segmentation fault when the "write same" command was executed in guest mode under the iSCSI protocol. This update fixes the regression and the "write same" command now functions in guest mode under iSCSI as intended.
Clone Of:
Environment:
Last Closed: 2015-03-05 08:05:21 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0349 0 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2015-03-05 12:27:34 UTC

Description Sibiao Luo 2014-04-02 08:17:30 UTC 
Description of problem:
Execute the WRITE SAME command in guest to the libiscsi LUN cause QEMU core dumped.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm
3.10.0-115.el7.x86_64
qemu-kvm-1.5.3-60.el7.x86_64
guest info:
# uname -r
3.10.0-115.el7.x86_64

How reproducible:
3/3

Steps to Reproduce:
1.prepare a libiscsi LUN.
# qemu-img info iscsi://10.66.8.202:3261/iqn.2003-01.org.linux-iscsi.dhcp-8-202.x8664:sn.10b139e7a94e/0
image: iscsi://10.66.8.202:3261/iqn.2003-01.org.linux-iscsi.dhcp-8-202.x8664:sn.10b139e7a94e/0
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: unavailable
2.start qemu with this libiscsi disk.
# /usr/libexec/qemu-kvm -M pc -cpu Conroe -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection...-drive file=iscsi://10.66.8.202:3261/iqn.2003-01.org.linux-iscsi.dhcp-8-202.x8664:sn.10b139e7a94e/0,if=none,media=disk,format=raw,id=drive-data-disk,cache=none,aio=native,discard=on -device virtio-scsi-pci,bus=pci.0,addr=0x7,id=scsi0,multifunction=on -device scsi-hd,drive=drive-data-disk,id=data-disk,bus=scsi0.0,logical_block_size=4096
3. Execute the following commands in guest:
# yes | head -n2048 > buf
# sg_write_same --in buf --num=32 --lba=80 /dev/sda
# sg_write_same --in /dev/zero --num=96 --lba=0 /dev/sda
# sg_write_same -U --in /dev/zero --num=16 --lba=0 /dev/sda
# time sg_write_same --in buf --num=65536 --lba=131074 /dev/sda
# time sg_write_same --in /dev/zero --num=65534 --lba=196608 /dev/sda
# sg_write_same --in /dev/zero --num=0 --lba=128 /dev/sda

Actual results:
after step 3, execute the time sg_write_same cause QEMU core dumped, i will attach the core dumped log later.
guest ]1.# yes | head -n2048 > buf
guest ]2.# sg_write_same --in buf --num=32 --lba=80 /dev/sda
guest ]3.# sg_write_same --in /dev/zero --num=96 --lba=0 /dev/sda
guest ]4.# sg_write_same -U --in /dev/zero --num=16 --lba=0 /dev/sda
guest ]5.# time sg_write_same --in buf --num=65536 --lba=131074 /dev/sda

real	0m0.225s
user	0m0.000s
sys	0m0.001s
guest ]6.# time sg_write_same --in /dev/zero --num=65534 --lba=196608 /dev/sda
Write failed: Broken pipe

(qemu) qemu-kvm: iSCSI: Failure. SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
Segmentation fault (core dumped)

Expected results:
It should no any core dumped occurs.

Additional info:
# /usr/libexec/qemu-kvm -M pc -cpu Conroe -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/RHEL-Server-7.0-64-virtio.qcow2bk,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,bus=pci.0,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:01:02:B6:40:21,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -k en-us -boot menu=on -qmp tcp:0:5555,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :2 -spice disable-ticketing,port=5932 -monitor stdio -drive file=iscsi://10.66.8.202:3261/iqn.2003-01.org.linux-iscsi.dhcp-8-202.x8664:sn.10b139e7a94e/0,if=none,media=disk,format=raw,id=drive-data-disk,cache=none,aio=native,discard=on -device virtio-scsi-pci,bus=pci.0,addr=0x7,id=scsi0,multifunction=on -device scsi-hd,drive=drive-data-disk,id=data-disk,bus=scsi0.0,logical_block_size=4096
Warning: option deprecated, use lost_tick_policy property of kvm-pit instead.
Comment 1 Sibiao Luo 2014-04-02 08:18:42 UTC 
Core was generated by `/usr/libexec/qemu-kvm -M pc -cpu Conroe -enable-kvm -m 2048 -smp 2,sockets=2,co'.
Program terminated with signal 11, Segmentation fault.
#0  iscsi_co_write_zeroes (bs=<optimized out>, sector_num=<optimized out>, nb_sectors=<optimized out>, 
    flags=<optimized out>) at block/iscsi.c:1023
1023	        if (iTask.status == SCSI_STATUS_CHECK_CONDITION &&

(gdb) bt
#0  iscsi_co_write_zeroes (bs=<optimized out>, sector_num=<optimized out>, nb_sectors=<optimized out>, 
    flags=<optimized out>) at block/iscsi.c:1023
#1  0x00007f65fecc1aff in bdrv_co_do_write_zeroes (bs=bs@entry=0x7f66001a5120, sector_num=sector_num@entry=1572864, 
    nb_sectors=nb_sectors@entry=32768, flags=flags@entry=BDRV_REQ_ZERO_WRITE) at block.c:2964
#2  0x00007f65fecc5d05 in bdrv_aligned_pwritev (flags=2, qiov=0x0, bytes=16777216, offset=<optimized out>, 
    req=0x7f66003d1480, bs=0x7f66001a5120) at block.c:3019
#3  bdrv_co_do_pwritev (bs=0x7f66001a5120, offset=<optimized out>, bytes=16777216, qiov=<optimized out>, 
    flags=BDRV_REQ_ZERO_WRITE) at block.c:3151
#4  0x00007f65fecc1aff in bdrv_co_do_write_zeroes (bs=bs@entry=0x7f66001a0ed0, sector_num=sector_num@entry=1572864, 
    nb_sectors=nb_sectors@entry=524272, flags=flags@entry=BDRV_REQ_ZERO_WRITE) at block.c:2964
#5  0x00007f65fecc5d05 in bdrv_aligned_pwritev (flags=2, qiov=0x0, bytes=268427264, offset=<optimized out>, 
    req=0x7f66003d1660, bs=0x7f66001a0ed0) at block.c:3019
#6  bdrv_co_do_pwritev (bs=0x7f66001a0ed0, offset=<optimized out>, bytes=268427264, qiov=<optimized out>, 
    flags=BDRV_REQ_ZERO_WRITE) at block.c:3151
#7  0x00007f65fecc6654 in bdrv_co_do_writev (flags=<optimized out>, qiov=<optimized out>, nb_sectors=<optimized out>, 
    sector_num=<optimized out>, bs=<optimized out>) at block.c:3175
#8  bdrv_co_do_rw (opaque=0x7f66005b9ab0) at block.c:4474
#9  0x00007f65fecfed2a in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at coroutine-ucontext.c:118
#10 0x00007f65f98ac570 in ?? () from /lib64/libc.so.6
#11 0x00007fff266fcd50 in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb) bt full
#0  iscsi_co_write_zeroes (bs=<optimized out>, sector_num=<optimized out>, nb_sectors=<optimized out>, 
    flags=<optimized out>) at block/iscsi.c:1023
        iscsilun = 0x7f66001a8350
        iTask = {status = 2, complete = 1, retries = 4, do_retry = 0, task = 0x0, co = 0x7f660028f950}
        lba = 1572864
        nb_blocks = <optimized out>
#1  0x00007f65fecc1aff in bdrv_co_do_write_zeroes (bs=bs@entry=0x7f66001a5120, sector_num=sector_num@entry=1572864, 
    nb_sectors=nb_sectors@entry=32768, flags=flags@entry=BDRV_REQ_ZERO_WRITE) at block.c:2964
        num = 32768
        drv = 0x7f65ff2bc740 <bdrv_iscsi>
        qiov = {iov = 0x11f883000, niov = -19928276, nalloc = 32613, size = 140076067394688}
        iov = {iov_base = 0x0, iov_len = 0}
        ret = -95
        max_write_zeroes = 32768
#2  0x00007f65fecc5d05 in bdrv_aligned_pwritev (flags=2, qiov=0x0, bytes=16777216, offset=<optimized out>, 
    req=0x7f66003d1480, bs=0x7f66001a5120) at block.c:3019
        waited = <optimized out>
        ret = <optimized out>
        nb_sectors = 32768
        drv = 0x7f65ff2bc740 <bdrv_iscsi>
        sector_num = 1572864
#3  bdrv_co_do_pwritev (bs=0x7f66001a5120, offset=<optimized out>, bytes=16777216, qiov=<optimized out>, 
    flags=BDRV_REQ_ZERO_WRITE) at block.c:3151
        req = {bs = 0x7f66001a5120, offset = 805306368, bytes = 16777216, is_write = true, serialising = false, 
          overlap_offset = 805306368, overlap_bytes = 16777216, list = {le_next = 0x0, le_prev = 0x7f66001a7310}, 
          co = 0x7f660028f950, wait_queue = {entries = {tqh_first = 0x0, tqh_last = 0x7f66003d14c0}, 
            ctx = 0x7f660000d090}, waiting_for = 0x0}
        align = <optimized out>
        head_buf = 0x0
        tail_buf = <optimized out>
        local_qiov = {iov = 0x7f660003d340, niov = 529010688, nalloc = 1, size = 4294975488}
        use_local_qiov = <optimized out>
        ret = <optimized out>
        __PRETTY_FUNCTION__ = "bdrv_co_do_pwritev"
#4  0x00007f65fecc1aff in bdrv_co_do_write_zeroes (bs=bs@entry=0x7f66001a0ed0, sector_num=sector_num@entry=1572864, 
    nb_sectors=nb_sectors@entry=524272, flags=flags@entry=BDRV_REQ_ZERO_WRITE) at block.c:2964
        num = 32768
        drv = 0x7f65ff2be560 <bdrv_raw>
        qiov = {iov = 0x0, niov = -19928276, nalloc = 32613, size = 140076067395168}
        iov = {iov_base = 0x0, iov_len = 0}
        ret = -95
        max_write_zeroes = 32768
#5  0x00007f65fecc5d05 in bdrv_aligned_pwritev (flags=2, qiov=0x0, bytes=268427264, offset=<optimized out>, 
    req=0x7f66003d1660, bs=0x7f66001a0ed0) at block.c:3019
        waited = <optimized out>
        ret = <optimized out>
        nb_sectors = 524272
        drv = 0x7f65ff2be560 <bdrv_raw>
        sector_num = 1572864
#6  bdrv_co_do_pwritev (bs=0x7f66001a0ed0, offset=<optimized out>, bytes=268427264, qiov=<optimized out>, 
    flags=BDRV_REQ_ZERO_WRITE) at block.c:3151
        req = {bs = 0x7f66001a0ed0, offset = 805306368, bytes = 268427264, is_write = true, serialising = false, 
          overlap_offset = 805306368, overlap_bytes = 268427264, list = {le_next = 0x0, le_prev = 0x7f66001a30c0}, 
          co = 0x7f660028f950, wait_queue = {entries = {tqh_first = 0x0, tqh_last = 0x7f66003d16a0}, 
            ctx = 0x7f660000d090}, waiting_for = 0x0}
        align = <optimized out>
        head_buf = 0x0
        tail_buf = <optimized out>
        local_qiov = {iov = 0x0, niov = 0, nalloc = 0, size = 0}
        use_local_qiov = <optimized out>
        ret = <optimized out>
        __PRETTY_FUNCTION__ = "bdrv_co_do_pwritev"
#7  0x00007f65fecc6654 in bdrv_co_do_writev (flags=<optimized out>, qiov=<optimized out>, nb_sectors=<optimized out>, 
    sector_num=<optimized out>, bs=<optimized out>) at block.c:3175
No locals.
#8  bdrv_co_do_rw (opaque=0x7f66005b9ab0) at block.c:4474
        acb = 0x7f66005b9ab0
        bs = <optimized out>
#9  0x00007f65fecfed2a in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at coroutine-ucontext.c:118
        self = 0x7f660028f950
        co = 0x7f660028f950
#10 0x00007f65f98ac570 in ?? () from /lib64/libc.so.6
No symbol table info available.
#11 0x00007fff266fcd50 in ?? ()
No symbol table info available.
#12 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) q
Comment 2 Sibiao Luo 2014-04-02 08:19:58 UTC 
# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 42
Model name:            Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Stepping:              7
CPU MHz:               1978.242
BogoMIPS:              6784.57
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7
Comment 3 Sibiao Luo 2014-04-02 08:43:28 UTC 
we support the fix WRITE SAME support in qemu-kvm-1.5.3-31.el7 according to bug 1007815.

If i use the qemu-kvm-1.5.3-31.el7.x86_64 which cann't execute the sg_write_same with the error message 'bad field in Write same(10) cdb, option probably not supported'.

qemu-kvm-1.5.3-31.el7.x86_64
guest ]# yes | head -n2048 > buf
guest ]# sg_write_same --in buf --num=32 --lba=80 /dev/sda
bad field in Write same(10) cdb, option probably not supported

qemu-kvm-1.5.3-32.el7.x86_64 ------- hit this issue
qemu-kvm-1.5.3-38.el7.x86_64 ------- hit this issue
qemu-kvm-1.5.3-59.el7.x86_64 ------- hit this issue
qemu-kvm-1.5.3-60.el7.x86_64 ------- hit this issue

Base on above, this issue is a regression issue caused by qemu-kvm-1.5.3-32.el7.x86_64.

Best Regards,
sluo
Comment 6 Paolo Bonzini 2014-04-02 13:45:51 UTC 
There are actually other problems.  One is the crash, which is fixed by upstream commit d9738fd2463f71530d8d92fbb52ebdd1d78074fc.

The second is that QEMU does not recognize INVALID_FIELD_IN_CDB (reported also by dmesg on the target as "WRITE_SAME sectors: 32768 exceeds max_write_same_len: 4096") and fall back to regular writes.  This will be fixed by commit d7b09f19fd85456ec282b03e736ce9244ff9acec (from scsi-next).

The third is that QEMU does not properly split the WRITE SAME requests because LBPME=0.  This is fixed by commit 24d3bd67aca958c8ea103646d9d326de00056e4d, which doesn't apply cleanly to RHEL but the conflict are easily solved, commit 5d259fc7da83249a4f78fe32de2bc2874a997a9f and commit e4942f2df8f9969231823fc51cedf8f042fe7a31 (this one from scsi-next).

There is also a fourth problem which is reported on the iSCSI target as "SCSI OP 8ah with too big sectors 32768 exceeds fabric_max_sectors: 8192".  I will open a separate bug for this once I figure out where the fix lies.  The above five patches are enough to fix this completely for WRITE SAME.
Comment 10 Fam Zheng 2014-04-04 07:01:14 UTC 
*** Bug 1025342 has been marked as a duplicate of this bug. ***
Comment 14 Miroslav Rezanina 2014-05-02 09:01:23 UTC 
Fix included in qemu-kvm-1.5.3-61.el7
Comment 17 Sibiao Luo 2014-10-13 06:46:16 UTC 
Verify this issue on qemu-kvm-1.5.3-75.el7.x86_64 which has no any core dumped.

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-171.el7.x86_64
qemu-kvm-1.5.3-75.el7.x86_64

Steps:
the same to comment #0.
after step 3, execute the time sg_write_same cause QEMU core dumped, i will attach the core dumped log later.
guest ]1.# yes | head -n2048 > buf
guest ]2.# sg_write_same --in buf --num=32 --lba=80 /dev/sdb
guest ]3.# sg_write_same --in /dev/zero --num=96 --lba=0 /dev/sdb
guest ]4.# sg_write_same -U --in /dev/zero --num=16 --lba=0 /dev/sdb
guest ]5.# time sg_write_same --in buf --num=65536 --lba=131074 /dev/sdb

real	0m0.399s
user	0m0.000s
sys	0m0.001s
guest ]6.# time sg_write_same --in /dev/zero --num=65534 --lba=196608 /dev/sdb

real	0m0.388s
user	0m0.000s
sys	0m0.003s

guest ]7.# sg_write_same --in /dev/zero --num=0 --lba=128 /dev/sdb
bad field in Write same(10) cdb, option probably not supported

(qemu) qemu-kvm: iSCSI: Failure. SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_OPERATION_CODE(0x2000)

(qemu) info status 
VM status: running

#############################################

Also tried on qemu-kvm-rhev-2.1.2-1.el7.x86_64 which has no any core dumped.

host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-183.el7.x86_64
qemu-kvm-rhev-2.1.2-1.el7.x86_64

Results:
# yes | head -n2048 > buf
# sg_write_same --in buf --num=32 --lba=80 /dev/sdb
# sg_write_same --in /dev/zero --num=96 --lba=0 /dev/sdb
# sg_write_same -U --in /dev/zero --num=16 --lba=0 /dev/sdb
# time sg_write_same --in buf --num=65536 --lba=131074 /dev/sdb

real	0m0.386s
user	0m0.000s
sys	0m0.002s
# time sg_write_same --in /dev/zero --num=65534 --lba=196608 /dev/sdb

real	0m0.371s
user	0m0.000s
sys	0m0.002s
# 
# sg_write_same --in /dev/zero --num=0 --lba=128 /dev/sdb
bad field in Write same(10) cdb, option probably not supported

(qemu) qemu-kvm: iSCSI Failure: SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_OPERATION_CODE(0x2000)

(qemu) info status 
VM status: running

Base on above, this issue has been fixed correctly, move to VERIFIED status.

Best Regards,
sluo
Comment 19 errata-xmlrpc 2015-03-05 08:05:21 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0349.html
Note You need to log in before you can comment on or make changes to this bug.