RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 956929 - /usr/libexec/qemu-kvm was killed by signal 6 (SIGABRT) when SCSI inquiry is sent to unsupported page inside the KVM guest
Summary: /usr/libexec/qemu-kvm was killed by signal 6 (SIGABRT) when SCSI inquiry is s...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.4
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Paolo Bonzini
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-04-26 02:55 UTC by Kenny Sun
Modified: 2018-12-04 15:16 UTC (History)
17 users (show)

Fixed In Version: qemu-kvm-0.12.1.2-2.413.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-11-21 06:54:29 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Crash reports generated by ABRT (1.80 MB, application/x-compressed)
2013-04-26 02:55 UTC, Kenny Sun 		no flags Details
Guest config XML (3.24 KB, text/xml)
2013-04-27 03:02 UTC, Kenny Sun 		no flags Details
Disk config XML (336 bytes, text/xml)
2013-04-27 03:02 UTC, Kenny Sun 		no flags Details
Verification logs 2013-10-18 (7.57 KB, text/plain)
2013-10-18 10:47 UTC, Kenny Sun 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2013:1553 0 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2013-11-20 21:40:29 UTC

Description Kenny Sun 2013-04-26 02:55:56 UTC 
Created attachment 740170 [details]
Crash reports generated by ABRT

Description of problem:

Inside the KVM guest, when a SCSI inquiry is sent for an unsupported page(e.g. 0xb2) to a SAN LUN attached through virtio-scsi, the /usr/libexec/qemu-kvm process gets killed from the host, and the guest is dead as a result.

Version-Release number of selected component (if applicable):

qemu-kvm-0.12.1.2-2.355.el6.x86_64

How reproducible:

From the guest
==============
[root@cdc-kvmguest2 ~]# /etc/vx/diag.d/vxscsiinq -d -e1 -p0x00
/dev/vx/rdmp/hitachi_usp-vm0_0a02

Inquiry for /dev/vx/rdmp/hitachi_usp-vm0_0a02, evpd 0x1, page code 0x0
Peripheral Qualifier/Device Type : 0
Number of supported page         : 6
Supported pages                  : 0 80 83 c0 e0 e3
        /dev/vx/rdmp/hitachi_usp-vm0_0a02: Raw data size 10
Bytes:   0 -   7    0x00  0x00  0x00  0x06  0x00  0x80  0x83  0xc0  ........
Bytes:   8 -   9    0xe0  0xe3   
//Note that the 0xb2 page is not supported
                                   ..
[root@cdc-kvmguest2 ~]# /etc/vx/diag.d/vxscsiinq -d -e1 -p0xb2 /dev/vx
/rdmp/hitachi_usp-vm0_0a02 <-- the guest is dead from here
Read from remote host 10.200.57.108: Connection timed out

From the host
=============
Process /usr/libexec/qemu-kvm was killed by signal 6 (SIGABRT)
  
Actual results:

The SCSI inquiry never returns, and the qemu-kvm process gets killed.


Expected results:

The SCSI inquiry should return an error or other message and it shouldn't lead to termination of the guest system.

Additional info:

LUNs are attached to the KVM guest using the following config:
<disk type='block' device='lun' sgio='unfiltered'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-path/pci-0000:04:00.0-fc-0x50060e8005655550-lun-20'/>
<target dev='sdm' bus='scsi'/>
<address type='drive' controller='1' bus='0' target='0' unit='0'/>
</disk>
Comment 1 Kenny Sun 2013-04-26 03:02:43 UTC 
* ABRT crash report is attached(abrt_report.tgz)
* I have removed the coredump file from the attached ABRT report due to large size. Please let know if it's needed for debugging and where to upload it.

The backtrace generated from the crash by ABRT
==============================================

[New Thread 23211]
[New Thread 23269]
[New Thread 23270]
[New Thread 24691]
[New Thread 23267]
[New Thread 23268]
[Thread debugging using libthread_db enabled]
Core was generated by `/usr/libexec/qemu-kvm -name guest2 -S -M rhel6.4.0 -enable-kvm -m 8192 -smp 4,s'.
Program terminated with signal 6, Aborted.
#0  0x00007f73c02a58a5 in raise () from /lib64/libc.so.6

Thread 6 (Thread 0x7f73b9faf700 (LWP 23268)):
#0  0x00007f73c0353a47 in ioctl () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f73c296642a in ?? ()
No symbol table info available.
#2  0x00007f73c29668d9 in ?? ()
No symbol table info available.
#3  0x00007f73c29677bd in ?? ()
No symbol table info available.
#4  0x00007f73c229c851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f73c035b90d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 5 (Thread 0x7f73ba9b0700 (LWP 23267)):
#0  0x00007f73c0353a47 in ioctl () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f73c296642a in ?? ()
No symbol table info available.
#2  0x00007f73c29668d9 in ?? ()
No symbol table info available.
#3  0x00007f73c29677bd in ?? ()
No symbol table info available.
#4  0x00007f73c229c851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f73c035b90d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 4 (Thread 0x7f71a1ffb700 (LWP 24691)):
#0  0x00007f73c22a07bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f73c2984187 in ?? ()
No symbol table info available.
#2  0x00007f73c229c851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00007f73c035b90d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 3 (Thread 0x7f73b8bad700 (LWP 23270)):
#0  0x00007f73c0353a47 in ioctl () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f73c296642a in ?? ()
No symbol table info available.
#2  0x00007f73c29668d9 in ?? ()
No symbol table info available.
#3  0x00007f73c29677bd in ?? ()
No symbol table info available.
#4  0x00007f73c229c851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f73c035b90d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 2 (Thread 0x7f73b95ae700 (LWP 23269)):
#0  0x00007f73c0353a47 in ioctl () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f73c296642a in ?? ()
No symbol table info available.
#2  0x00007f73c29668d9 in ?? ()
No symbol table info available.
#3  0x00007f73c29677bd in ?? ()
No symbol table info available.
#4  0x00007f73c229c851 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f73c035b90d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 1 (Thread 0x7f73c288b980 (LWP 23211)):
#0  0x00007f73c02a58a5 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f73c02a7085 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007f73c029ea1e in __assert_fail_base () from /lib64/libc.so.6
No symbol table info available.
#3  0x00007f73c029eae0 in __assert_fail () from /lib64/libc.so.6
No symbol table info available.
#4  0x00007f73c29ba555 in ?? ()
No symbol table info available.
#5  0x00007f73c29be9e2 in ?? ()
No symbol table info available.
#6  0x00007f73c2984430 in ?? ()
No symbol table info available.
#7  0x00007f73c29844f8 in ?? ()
No symbol table info available.
#8  0x00007f73c294229f in ?? ()
No symbol table info available.
#9  0x00007f73c296497a in ?? ()
No symbol table info available.
#10 0x00007f73c2945008 in main ()
No symbol table info available.
From                To                  Syms Read   Shared Object Library
0x00007f73c24b4140  0x00007f73c24b74f8  Yes (*)     /lib64/librt.so.1
0x00007f73c229a660  0x00007f73c22a5eb8  Yes (*)     /lib64/libpthread.so.0
0x00007f73c1fc3470  0x00007f73c203a0c8  Yes (*)     /lib64/libglib-2.0.so.0
0x00007f73c1daf570  0x00007f73c1daf721  Yes (*)     /lib64/libaio.so.1
0x00007f73c1baa630  0x00007f73c1bad4a8  Yes (*)     /usr/lib64/libusbredirparser.so.1
0x00007f73c19a6e10  0x00007f73c19a7688  Yes (*)     /lib64/libutil.so.1
0x00007f73c16e8de0  0x00007f73c1769d68  Yes (*)     /lib64/libasound.so.2
0x00007f73c1483e50  0x00007f73c14ab3d8  Yes (*)     /usr/lib64/libpulse.so.0
0x00007f73c1276490  0x00007f73c12776a8  Yes (*)     /usr/lib64/libpulse-simple.so.0
0x00007f73c105f6e0  0x00007f73c106f578  Yes (*)     /usr/lib64/libsasl2.so.2
0x00007f73c0dcdf30  0x00007f73c0e39568  Yes (*)     /usr/lib64/libgnutls.so.26
0x00007f73c0aa83c0  0x00007f73c0b765a8  Yes (*)     /usr/lib64/libspice-server.so.1
0x00007f73c081fe70  0x00007f73c085fec8  Yes (*)     /lib64/libm.so.6
0x00007f73c0608120  0x00007f73c06133a8  Yes (*)     /lib64/libz.so.1
0x00007f73c0291a20  0x00007f73c03b272c  Yes (*)     /lib64/libc.so.6
0x00007f73c26bab00  0x00007f73c26d385b  Yes (*)     /lib64/ld-linux-x86-64.so.2
0x00007f73c006fde0  0x00007f73c0070998  Yes (*)     /lib64/libdl.so.2
0x00007f73bfe2f4a0  0x00007f73bfe5a898  Yes (*)     /usr/lib64/libpulsecommon-0.9.21.so
0x00007f73bfb00cf0  0x00007f73bfb8ad38  Yes (*)     /usr/lib64/libX11.so.6
0x00007f73bf8dca40  0x00007f73bf8e10c8  Yes (*)     /usr/lib64/libSM.so.6
0x00007f73bf6c3d70  0x00007f73bf6d27a8  Yes (*)     /usr/lib64/libICE.so.6
0x00007f73bf4ba3b0  0x00007f73bf4bd3b8  Yes (*)     /usr/lib64/libXtst.so.6
0x00007f73bf2b0f10  0x00007f73bf2b4ab8  Yes (*)     /lib64/libwrap.so.0
0x00007f73bf04f560  0x00007f73bf090df8  Yes (*)     /usr/lib64/libsndfile.so.1
0x00007f73bee461c0  0x00007f73bee487e8  Yes (*)     /usr/lib64/libasyncns.so.0
0x00007f73bec0b1c0  0x00007f73bec327b8  Yes (*)     /lib64/libdbus-1.so.3
0x00007f73be9ed930  0x00007f73be9fc938  Yes (*)     /lib64/libresolv.so.2
0x00007f73be7b3c00  0x00007f73be7b89a8  Yes (*)     /lib64/libcrypt.so.1
0x00007f73be5a4a50  0x00007f73be5aff08  Yes (*)     /usr/lib64/libtasn1.so.3
0x00007f73be334e00  0x00007f73be37e278  Yes (*)     /lib64/libgcrypt.so.11
0x00007f73be11fe40  0x00007f73be12afb8  Yes (*)     /usr/lib64/libcelt051.so.0
0x00007f73bded2d20  0x00007f73bdf05fa8  Yes (*)     /usr/lib64/libjpeg.so.62
0x00007f73bdc51020  0x00007f73bdcbc268  Yes (*)     /usr/lib64/libpixman-1.so.0
0x00007f73bda00570  0x00007f73bda320c8  Yes (*)     /usr/lib64/libssl.so.10
0x00007f73bd6aea00  0x00007f73bd7761c8  Yes (*)     /usr/lib64/libcrypto.so.10
0x00007f73bd43d860  0x00007f73bd449938  Yes (*)     /usr/lib64/libxcb.so.1
0x00007f73bd2315a0  0x00007f73bd232cc8  Yes (*)     /lib64/libuuid.so.1
0x00007f73bd0206c0  0x00007f73bd02bfc8  Yes (*)     /usr/lib64/libXext.so.6
0x00007f73bce10030  0x00007f73bce1a338  Yes (*)     /usr/lib64/libXi.so.6
0x00007f73bcbf9070  0x00007f73bcc069f8  Yes (*)     /lib64/libnsl.so.1
0x00007f73bc9c6e70  0x00007f73bc9ebe08  Yes (*)     /usr/lib64/libFLAC.so.8
0x00007f73bc5f7a30  0x00007f73bc5f9ed8  Yes (*)     /usr/lib64/libvorbisenc.so.2
0x00007f73bc3b8ae0  0x00007f73bc3cf9b8  Yes (*)     /usr/lib64/libvorbis.so.0
0x00007f73bc1b08d0  0x00007f73bc1b2c28  Yes (*)     /usr/lib64/libogg.so.0
0x00007f73bbf502b0  0x00007f73bbf8f078  Yes (*)     /lib64/libfreebl3.so
0x00007f73bbd49820  0x00007f73bbd49d88  Yes (*)     /lib64/libgpg-error.so.0
0x00007f73bbb0fc30  0x00007f73bbb3d728  Yes (*)     /lib64/libgssapi_krb5.so.2
0x00007f73bb83a430  0x00007f73bb8b3a38  Yes (*)     /lib64/libkrb5.so.3
0x00007f73bb61c3f0  0x00007f73bb61cfc8  Yes (*)     /lib64/libcom_err.so.2
0x00007f73bb3f33d0  0x00007f73bb40c5a8  Yes (*)     /lib64/libk5crypto.so.3
0x00007f73bb1ecdd0  0x00007f73bb1edb58  Yes (*)     /usr/lib64/libXau.so.6
0x00007f73bafe3a40  0x00007f73bafe90c8  Yes (*)     /lib64/libkrb5support.so.0
0x00007f73baddebf0  0x00007f73baddf1d8  Yes (*)     /lib64/libkeyutils.so.1
0x00007f73babc4850  0x00007f73babd4c78  Yes (*)     /lib64/libselinux.so.1
0x00007f73ba9b31f0  0x00007f73ba9bb648  Yes (*)     /lib64/libnss_files.so.2
0x00007f73a9df7040  0x00007f73a9df8d08  Yes (*)     /usr/lib64/sasl2/libanonymous.so
0x00007f73a9beaa00  0x00007f73a9bf2698  Yes (*)     /usr/lib64/sasl2/libdigestmd5.so
0x00007f73a99e43a0  0x00007f73a99e7018  Yes (*)     /usr/lib64/sasl2/libsasldb.so
0x00007f73a96976a0  0x00007f73a97ad9f8  Yes (*)     /lib64/libdb-4.7.so
0x00007f73a946a150  0x00007f73a946c638  Yes (*)     /usr/lib64/sasl2/libcrammd5.so
0x00007f73a9262890  0x00007f73a9266be8  Yes (*)     /usr/lib64/sasl2/libgssapiv2.so
0x00007f73a905d040  0x00007f73a905eea8  Yes (*)     /usr/lib64/sasl2/liblogin.so
0x00007f73a8e58050  0x00007f73a8e59ea8  Yes (*)     /usr/lib64/sasl2/libplain.so
0x00007fff89dff700  0x00007fff89dffaa0  Yes         /lib/modules/2.6.32-358.el6.x86_64/vdso/vdso.so
(*): Shared library is missing debugging information.
$1 = 0xffffffffc28a4000 <Address 0xffffffffc28a4000 out of bounds>
No symbol "__glib_assert_msg" in current context.
rax            0x0	0
rbx            0x7f73c28a4000	140135161806848
rcx            0xffffffffffffffff	-1
rdx            0x6	6
rsi            0x5aab	23211
rdi            0x5aab	23211
rbp            0x7f73c2afe068	0x7f73c2afe068
rsp            0x7fff89dddd48	0x7fff89dddd48
r8             0xfefefefefefefeff	-72340172838076673
r9             0xfefefefefefeff09	-72340172838076663
r10            0x8	8
r11            0x202	514
r12            0x7f73c2afe0c0	140135164272832
r13            0x7f73c2afdfa0	140135164272544
r14            0x1	1
r15            0x1	1
rip            0x7f73c02a58a5	0x7f73c02a58a5 <raise+53>
eflags         0x202	[ IF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
Dump of assembler code for function raise:
   0x00007f73c02a5870 <+0>:	mov    %fs:0x2d4,%eax
   0x00007f73c02a5878 <+8>:	mov    %fs:0x2d0,%esi
   0x00007f73c02a5880 <+16>:	test   %esi,%esi
   0x00007f73c02a5882 <+18>:	jne    0x7f73c02a58b0 <raise+64>
   0x00007f73c02a5884 <+20>:	mov    $0xba,%eax
   0x00007f73c02a5889 <+25>:	syscall 
   0x00007f73c02a588b <+27>:	mov    %eax,%esi
   0x00007f73c02a588d <+29>:	mov    %eax,%fs:0x2d0
   0x00007f73c02a5895 <+37>:	movslq %edi,%rdx
   0x00007f73c02a5898 <+40>:	movslq %esi,%rsi
   0x00007f73c02a589b <+43>:	movslq %eax,%rdi
   0x00007f73c02a589e <+46>:	mov    $0xea,%eax
   0x00007f73c02a58a3 <+51>:	syscall 
=> 0x00007f73c02a58a5 <+53>:	cmp    $0xfffffffffffff000,%rax
   0x00007f73c02a58ab <+59>:	ja     0x7f73c02a58bf <raise+79>
   0x00007f73c02a58ad <+61>:	repz retq 
   0x00007f73c02a58af <+63>:	nop
   0x00007f73c02a58b0 <+64>:	test   %eax,%eax
   0x00007f73c02a58b2 <+66>:	jg     0x7f73c02a5895 <raise+37>
   0x00007f73c02a58b4 <+68>:	test   $0x7fffffff,%eax
   0x00007f73c02a58b9 <+73>:	jne    0x7f73c02a58cf <raise+95>
   0x00007f73c02a58bb <+75>:	mov    %esi,%eax
   0x00007f73c02a58bd <+77>:	jmp    0x7f73c02a5895 <raise+37>
   0x00007f73c02a58bf <+79>:	mov    0x35a6da(%rip),%rdx        # 0x7f73c05fffa0
   0x00007f73c02a58c6 <+86>:	neg    %eax
   0x00007f73c02a58c8 <+88>:	mov    %eax,%fs:(%rdx)
   0x00007f73c02a58cb <+91>:	or     $0xffffffffffffffff,%eax
   0x00007f73c02a58ce <+94>:	retq   
   0x00007f73c02a58cf <+95>:	neg    %eax
   0x00007f73c02a58d1 <+97>:	jmp    0x7f73c02a5895 <raise+37>
End of assembler dump.
Comment 2 Sibiao Luo 2013-04-26 09:44:01 UTC 
Hi Kenny Sun,

    Thanks for your opening this bug. vxscsiinq command is provided by 3rd party tool to send scsi command. I tried our internal open source tools(sg3_utils) whether have this issue, but did not met this issue, please help check it and paste your opinion here. Could you tell where can get or download the official 3rd party tool package which provide the vxscsiinq command. 

host info:
kernel-2.6.32-369.el6.x86_64
qemu-kvm-0.12.1.2-2.360.el6.x86_64
seabios-0.6.1.2-27.el6.x86_64
guest info:
kernel-2.6.32-369.el6.x86_64

Steps:
1.boot up a guest passthrough a iSCSI SAN LUN with virtio-scsi interface.
e.g:# /usr/libexec/qemu-kvm -S -M rhel6.4.0 -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -name sluo-test -uuid `uuidgen` -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/RHEL-Server-6.4-64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-system-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2e:5f:0a:1d:b1,bus=pci.0,addr=0x5,bootindex=2,ioeventfd=off -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -qmp tcp:0:4444,server,nowait -k en-us -boot menu=on -vnc :2 -spice disable-ticketing,port=5932 -vga qxl -monitor stdio -drive file=/dev/disk/by-path/ip-10.66.90.100\:3260-iscsi-iqn.2001-05.com.equallogic\:0-8a0906-0971f7d03-1dff49b26885073d-s2-sluo-172259-lun-0,if=none,id=drive-data-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x7,id=scsi0 -device scsi-block,bus=scsi0.0,drive=drive-data-disk,id=data-disk
2.install the sg3_utils RPM in guest.
3.sends SCSI INQUIRY to the disk.
# sg_inq /dev/sda
standard INQUIRY:
  PQual=0  Device_type=0  RMB=0  version=0x05  [SPC-3]
  [AERC=0]  [TrmTsk=0]  NormACA=1  HiSUP=1  Resp_data_format=2
  SCCS=0  ACC=0  TPGS=1  3PC=1  Protect=0  BQue=0
  EncServ=0  MultiP=0  [MChngr=0]  [ACKREQQ=0]  Addr16=0
  [RelAdr=0]  WBus16=0  Sync=0  Linked=0  [TranDis=0]  CmdQue=1
  [SPI: Clocking=0x0  QAS=0  IUS=0]
    length=66 (0x42)   Peripheral device type: disk
 Vendor identification: EQLOGIC 
 Product identification: 100E-00         
 Product revision level: 5.0 
 Unit serial number: 6090A038D0F771093D078568B249FF1D
4.list available VPD pages.
# sg_inq -e /dev/sda
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0	Supported VPD pages
     0x80	Unit serial number
     0x83	Device identification
     0xb2	Thin provisioning (sbc3)
     0xc0	vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xc1	vendor: Date code (seagate)
     0xc2	vendor: Jumper settings (seagate); Software version (RDAC)
5.send a support page to it.
# sg_inq -H -p 0x0 /dev/sda
VPD INQUIRY, page code=0x00:
 00     00 00 00 07 00 80 83 b2  c0 c1 c2                   ...........     
6.send a unsupport page to it.
# sg_inq -H -p 0x1 /dev/sda
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)

Result:
after step 6, SCSI inquiry prompted an 'page not supported' message and qemu-kvm/VM work well.


Best Regards.
sluo
Comment 3 Kenny Sun 2013-04-26 10:03:18 UTC 
Hi Sibiao,

Thanks for quickly looking into it! Given that the issue is not reproduciple using another SCSI inquiry tool, I will first work with our own engineering team to do some comparing tests, hopefully to further narrow down the issue.

Regarding the vxscsiinq binary, I will check with our dev team to see if I can provide it to you guys.

Will keep you posted.

Thanks,
Kenny
Comment 4 Kenny Sun 2013-04-27 03:01:43 UTC 
Hi Sibiao,

I have done the same tests on my test bed using the sg_inq command, and the issue is reproducible.

[root@cdc-kvmguest1 ~]# lsscsi
[2:0:0:87]   disk    HITACHI  OPEN-V           6008  /dev/sdb
[3:0:0:0]    disk    QEMU     QEMU HARDDISK    0.12  /dev/sda
[root@cdc-kvmguest1 ~]# sg_inq -e /dev/sdb
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0        Supported VPD pages
     0x80       Unit serial number
     0x83       Device identification
     0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xe0
     0xe3
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0x0 /dev/sdb
VPD INQUIRY, page code=0x00:
 00     00 00 00 06 00 80 83 c0  e0 e3                      ..........
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:  <=== Crash occurred here.

Could it be due to the guest config difference? I'm attaching the guest config xml along with the disk device xml files for a reference.

I started the guest without the sdb disk, and then attach the disk using attach-device disk.xml.


Thanks,
Kenny
Comment 5 Kenny Sun 2013-04-27 03:02:32 UTC 
Created attachment 740668 [details]
Guest config XML
Comment 6 Kenny Sun 2013-04-27 03:02:58 UTC 
Created attachment 740669 [details]
Disk config XML
Comment 7 Sibiao Luo 2013-04-27 06:33:56 UTC 
(In reply to comment #4)
> Hi Sibiao,
> 
> I have done the same tests on my test bed using the sg_inq command, and the
> issue is reproducible.
> 
> [root@cdc-kvmguest1 ~]# lsscsi
> [2:0:0:87]   disk    HITACHI  OPEN-V           6008  /dev/sdb
> [3:0:0:0]    disk    QEMU     QEMU HARDDISK    0.12  /dev/sda
> [root@cdc-kvmguest1 ~]# sg_inq -e /dev/sdb
> VPD INQUIRY, page code=0x00:
>    [PQual=0  Peripheral device type: disk]
>    Supported VPD pages:
>      0x0        Supported VPD pages
>      0x80       Unit serial number
>      0x83       Device identification
>      0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
>      0xe0
>      0xe3
> [root@cdc-kvmguest1 ~]# sg_inq -H -p 0x0 /dev/sdb
> VPD INQUIRY, page code=0x00:
>  00     00 00 00 06 00 80 83 c0  e0 e3                      ..........
> [root@cdc-kvmguest1 ~]# sg_inq -H -p 0x01 /dev/sdb
> VPD INQUIRY, page code=0x01:  <=== Crash occurred here.
> 
> Could it be due to the guest config difference? I'm attaching the guest
> config xml along with the disk device xml files for a reference.
> 
> I started the guest without the sdb disk, and then attach the disk using
> attach-device disk.xml.
> 
hmm, I retested it with pure qemu-kvm command line, and anther guy also tried it with virt-manager tools, all of the two methods did not met this issue. Did you have other actions in guest? or your image was damaged, would you help to retest it with a new image? and show your qemu-kvm command line var '# ps axu | grep qemu-kvm' here.

- qemu-kvm command test:
# /usr/libexec/qemu-kvm -S -M rhel6.4.0 -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -name sluo-test -uuid `uuidgen` -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/RHEL-Server-6.4-64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x3,id=scsi0 -device scsi-hd,bus=scsi0.0,drive=drive-system-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2e:5f:0a:1d:b1,bus=pci.0,addr=0x4,bootindex=2,ioeventfd=off -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x5 -qmp tcp:0:4444,server,nowait -k en-us -boot menu=on -vnc :2 -spice disable-ticketing,port=5932 -vga qxl -monitor stdio
(qemu) device_add virtio-scsi-pci,bus=pci.0,addr=0x6,id=scsi1
(qemu) __com.redhat_drive_add file=/dev/disk/by-path/ip-10.66.90.100:3260-iscsi-iqn.2001-05.com.equallogic:0-8a0906-0971f7d03-1dff49b26885073d-s2-sluo-172259-lun-0,id=drive-data-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop
(qemu) device_add scsi-block,bus=scsi1.0,drive=drive-data-disk,id=data-disk
# lsscsi
[1:0:0:0]    cd/dvd  QEMU     QEMU DVD-ROM     0.12  /dev/sr0 
[2:0:0:0]    disk    QEMU     QEMU HARDDISK    0.12  /dev/sda 
[3:0:0:0]    disk    EQLOGIC  100E-00          5.0   /dev/sdb 
# sg_inq -e /dev/sdb
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0	Supported VPD pages
     0x80	Unit serial number
     0x83	Device identification
     0xb2	Thin provisioning (sbc3)
     0xc0	vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xc1	vendor: Date code (seagate)
     0xc2	vendor: Jumper settings (seagate); Software version (RDAC)
# sg_inq -H -p 0x0 /dev/sdb
VPD INQUIRY, page code=0x00:
 00     00 00 00 07 00 80 83 b2  c0 c1 c2                   ...........     
# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)
# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)
# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)
...

- virt-manager tools test:
use virt-manager to boot up a guest, then attach a iSCSI LUN to the guest with the tools, then it will suggestion to reboot VM. Then do the same testing as above, did not met this issue.

Best Regards.
sluo
Comment 9 Kenny Sun 2013-04-27 08:42:07 UTC 
Hi Sibiao,

I have done some further testing with another disk enclosure, and the issue is not reproducible.

[root@cdc-kvmguest1 ~]# lsscsi
[2:0:0:0]    disk    QEMU     QEMU HARDDISK    0.12  /dev/sda
[2:0:0:87]   disk    HITACHI  OPEN-V           6008  /dev/sdb
[2:0:0:88]   disk    HP       HSV340           1100  /dev/sdc

With HP P6350
==================
[root@cdc-kvmguest1 ~]# sg_inq -e /dev/sdc
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0        Supported VPD pages
     0x1
     0x80       Unit serial number
     0x81       Implemented operating definitions (obsolete)
     0x82       ASCII implemented operating definition (obsolete)
     0x83       Device identification
     0x86       Extended INQUIRY data
     0x87       Mode page policy
     0xb0       Block limits (sbc2)
     0xb2       Thin provisioning (sbc3)
     0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xd0
     0xed
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0x02 /dev/sdc
VPD INQUIRY, page code=0x02:
    inquiry: field in cdb illegal (page not supported)
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0x01 /dev/sdc
VPD INQUIRY, page code=0x01:
 00     00 01 00 01 00                                      .....
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0xb1 /dev/sdc
VPD INQUIRY, page code=0xb1:
    inquiry: field in cdb illegal (page not supported)


With Hitachi USPVM
==================
[root@cdc-kvmguest1 ~]# sg_inq -e /dev/sdb
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0        Supported VPD pages
     0x80       Unit serial number
     0x83       Device identification
     0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xe0
     0xe3
[root@cdc-kvmguest1 ~]# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:  <===== Crash is hit.

I guess the issue is HW specific. However, the same command could return error on a physical host with the same LUN.

On the physical host
====================
# ll /dev/disk/by-path/pci-0000:04:00.1-fc-0x50060e8005655570-lun-28
lrwxrwxrwx 1 root root 10 Apr 23 23:48 /dev/disk/by-path/pci-0000:04:00.1-fc-0x50060e8005655570-lun-28 -> ../../sdbg
# sg_inq -e /dev/sdbg
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0        Supported VPD pages
     0x80       Unit serial number
     0x83       Device identification
     0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xe0
     0xe3
# sg_inq -H -p 0xb2 /dev/sdbg
VPD INQUIRY, page code=0xb2:
    inquiry: field in cdb illegal (page not supported)


Please refer to the qemu-kvm cmd-line below:

# ps axu | grep qemu-kvm
qemu      4619 33.5  5.6 17427172 1404244 ?    Sl   01:32   0:45 /usr/libexec/qemu-kvm -name guest1 -S -M rhel6.4.0 -enable-kvm -m 16005 -smp 2,sockets=2,cores=1,threads=1 -uuid 31dde40b-72ad-e8d8-36ed-6c69f21dc85b -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/guest1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -drive file=/var/lib/libvirt/images/guest1.img,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:31:d5:1b,bus=pci.0,addr=0x3 -netdev tap,fd=28,id=hostnet1 -device rtl8139,netdev=hostnet1,id=net1,mac=52:54:00:90:8c:94,bus=pci.0,addr=0xb -netdev tap,fd=29,id=hostnet2 -device rtl8139,netdev=hostnet2,id=net2,mac=52:54:00:e0:60:80,bus=pci.0,addr=0xc -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -k en-us -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
root      4796  0.0  0.0 103244   848 pts/1    S+   01:35   0:00 grep qemu-kvm
Comment 10 Paolo Bonzini 2013-04-29 16:59:22 UTC 
Hi Kenny, please run the sg_inq command on the host and also provide a "strace" of the sg_inq command.

Thanks!
Comment 11 Paolo Bonzini 2013-04-29 17:00:14 UTC 
Also please install the debuginfo package and reproduce it using qemu-kvm.
Comment 12 Richard Fan 2013-05-02 02:08:20 UTC 
HI Paolo,

Kenny is OOO, and I will help during his vercation.

[root@cdc-r710s2 ~]# strace sg_inq -H -p 0xb2 /dev/sdbg
execve("/usr/bin/sg_inq", ["sg_inq", "-H", "-p", "0xb2", "/dev/sdbg"], [/* 31 vars */]) = 0
brk(0)                                  = 0x1086000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc641ce000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=215683, ...}) = 0
mmap(NULL, 215683, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fbc64199000
close(3)                                = 0
open("/usr/lib64/libsgutils2.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \223\0;5\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=159288, ...}) = 0
mmap(0x353b000000, 2252096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x353b000000
mprotect(0x353b022000, 2093056, PROT_NONE) = 0
mmap(0x353b221000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21000) = 0x353b221000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\1:5\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1922152, ...}) = 0
mmap(0x353a000000, 3745960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x353a000000
mprotect(0x353a18a000, 2093056, PROT_NONE) = 0
mmap(0x353a389000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x189000) = 0x353a389000
mmap(0x353a38e000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x353a38e000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc64198000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc64197000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc64196000
arch_prctl(ARCH_SET_FS, 0x7fbc64197700) = 0
mprotect(0x353a389000, 16384, PROT_READ) = 0
mprotect(0x3539a1f000, 4096, PROT_READ) = 0
munmap(0x7fbc64199000, 215683)          = 0
brk(0)                                  = 0x1086000
brk(0x10a7000)                          = 0x10a7000
open("/proc/devices", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc641cd000
read(3, "Character devices:\n  1 mem\n  4 /"..., 1024) = 544
close(3)                                = 0
munmap(0x7fbc641cd000, 4096)            = 0
open("/dev/sdbg", O_RDONLY|O_NONBLOCK)  = 3
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc641cd000
write(1, "VPD INQUIRY, page code=0xb2:\n", 29VPD INQUIRY, page code=0xb2:
) = 29
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(67, 160), ...}) = 0
ioctl(3, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 01, b2, 00, fc, 00], mx_sb_len=32, iovec_count=0, dxfer_len=252, timeout=60000, flags=0, data[252]=["\177\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...], status=02, masked_status=01, sb[32]=[70, 00, 05, 00, 00, 00, 00, 58, 00, 00, 00, 00, 24, 00, 00, 00, 00, 00, 12, 80, d4, 22, d4, 22, 00, 01, 02, 02, 00, 1c, 0f, b5], host_status=0, driver_status=0x8, resid=252, duration=1, info=0x1}) = 0
write(2, "    inquiry: field in cdb illega"..., 55    inquiry: field in cdb illegal (page not supported)
) = 55
close(3)                                = 0
exit_group(5)                           = ?
[root@cdc-r710s2 ~]# 

Thanks,
Richard.
Comment 13 Richard Fan 2013-05-02 02:16:09 UTC 
Hi Paolo,

For comment item 11. I failed to download the dubuginfo package from RHEL website.
So can I ask for your kinldy help to point me where I can get it?

Regards,
Richard.
Comment 14 Richard Fan 2013-05-03 08:09:14 UTC 
Hi Paolo,

Do you have any update on this case? Please let me know if you want to collect more information.

Thanks,
Richard.
Comment 15 Sibiao Luo 2013-05-06 08:24:14 UTC 
Our lab only has Dell and Netapp storage, have no Hitachi storage.

I tested the 'Dell PS5000E' and 'Netapp FAS2050' iSCSI storage device LUN with the same steps as comment #7. Both of them are work well, have no such issue.

host info:
kernel-2.6.32-374.el6.x86_64
qemu-kvm-0.12.1.2-2.362.el6.x86_64
geust info:
kernel-2.6.32-374.el6.x86_64

Best Regards,
sluo
Comment 16 Richard Fan 2013-05-06 08:39:17 UTC 
Hi Sluo,

The issue can not be seen in with another HP array too.

So it might be array specific issue.

[root@cdc-kvmguest2 ~]# lsscsi
[6:0:0:0]    disk    HP       HSV340           1100  /dev/sda 
root@cdc-kvmguest2 ~]# sg_inq -e /dev/sda
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0        Supported VPD pages
     0x1
     0x80       Unit serial number
     0x81       Implemented operating definitions (obsolete)
     0x82       ASCII implemented operating definition (obsolete)
     0x83       Device identification
     0x86       Extended INQUIRY data
     0x87       Mode page policy
     0xb0       Block limits (sbc2)
     0xb2       Thin provisioning (sbc3)
     0xc0       vendor: Firmware numbers (seagate); Unit path report (EMC)
     0xd0
     0xed

[root@cdc-kvmguest2 ~]#  sg_inq -H -p 0x02 /dev/sda
VPD INQUIRY, page code=0x02:
    inquiry: field in cdb illegal (page not supported)

[root@cdc-kvmguest2 ~]# sg_inq -H -p 0x01 /dev/sda
VPD INQUIRY, page code=0x01:
 00     00 01 00 01 00                                      .....  
Thanks,
Richard.
Comment 17 Richard Fan 2013-05-06 09:29:37 UTC 
Hi Sluo,

Where can I download the kernel package and qemu-kvm package?

I can have a try with our HDS uspv array.

Thanks,
Richard.
Comment 18 Sibiao Luo 2013-05-06 09:44:15 UTC 
(In reply to comment #17)
> Hi Sluo,
> 
> Where can I download the kernel package and qemu-kvm package?
> 
> I can have a try with our HDS uspv array.
> 
You can get it from your system ISO or DVD/CD ROM file(rhel6.4 release OS), as it can be reproduced in rhel6.4 release(comment #0: qemu-kvm-0.12.1.2-2.355.el6.x86_64).
Or you can let Paolo give you the dubuginfo package.

Best Regards,
sluo
Comment 19 Richard Fan 2013-05-06 09:48:13 UTC 
Hi Sluo,

Sorry, I mean where can I get the packages you mentioned？
I only have the GA version.


kernel-2.6.32-374.el6.x86_64
qemu-kvm-0.12.1.2-2.362.el6.x86_64


Thanks,
Richard.
Comment 20 Sibiao Luo 2013-05-06 10:03:07 UTC 
(In reply to comment #19)
> Hi Sluo,
> 
> Sorry, I mean where can I get the packages you mentioned？
> I only have the GA version.
> 
> kernel-2.6.32-374.el6.x86_64
> qemu-kvm-0.12.1.2-2.362.el6.x86_64
> 
rhel6.5 is internally patch now which is unstable and not released.
rhel6.4GA and rhel6.5 both can hit this issue, so I think you can just install rhel6.4GA patch to get the logs, this issue is not particular for rhel6.5.
Comment 21 Richard Fan 2013-05-06 10:06:49 UTC 
(In reply to comment #20)
> (In reply to comment #19)
> > Hi Sluo,
> > 
> > Sorry, I mean where can I get the packages you mentioned？
> > I only have the GA version.
> > 
> > kernel-2.6.32-374.el6.x86_64
> > qemu-kvm-0.12.1.2-2.362.el6.x86_64
> > 
> rhel6.5 is internally patch now which is unstable and not released.
> rhel6.4GA and rhel6.5 both can hit this issue, so I think you can just
> install rhel6.4GA patch to get the logs, this issue is not particular for
> rhel6.5.

Hi Sluo,

Thanks for your reply.

I am able to download the debuginfo package now. and will try to reproduce the issue as per Paolo's comment.

Regards,
Richard.
Comment 22 Paolo Bonzini 2013-05-06 15:42:35 UTC 
Richard, if possible please save a core file of QEMU and pass it to me through any means you see fit (not email :)).
Comment 23 Richard Fan 2013-05-07 02:06:47 UTC 
(In reply to comment #22)
> Richard, if possible please save a core file of QEMU and pass it to me
> through any means you see fit (not email :)).

Hi Paolo,

debuginfo packages in host:

kernel-debuginfo-common-x86_64-2.6.32-358.el6.x86_64
kernel-debuginfo-2.6.32-358.el6.x86_64
qemu-kvm-debuginfo-0.12.1.2-2.355.el6.x86_64

And debuginfo packages in kvm-guest:

kernel-debuginfo-common-x86_64-2.6.32-358.el6.x86_64
kernel-debuginfo-2.6.32-358.el6.x86_64

##
I reproduced the issue on kvm-guest:

[root@cdc-kvmguest2 ~]# strace sg_inq -H -p 0xb2 /dev/sdb
execve("/usr/bin/sg_inq", ["sg_inq", "-H", "-p", "0xb2", "/dev/sdb"], [/* 32 vars */]) = 0
brk(0)                                  = 0x2238000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9fd1000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=215683, ...}) = 0
mmap(NULL, 215683, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f1ed9f9c000
close(3)                                = 0
open("/usr/lib64/libsgutils2.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \223\200\273>\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=159288, ...}) = 0
mmap(0x3ebb800000, 2252096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3ebb800000
mprotect(0x3ebb822000, 2093056, PROT_NONE) = 0
mmap(0x3ebba21000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21000) = 0x3ebba21000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\201\272>\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1922152, ...}) = 0
mmap(0x3eba800000, 3745960, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3eba800000
mprotect(0x3eba98a000, 2093056, PROT_NONE) = 0
mmap(0x3ebab89000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x189000) = 0x3ebab89000
mmap(0x3ebab8e000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3ebab8e000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9f9b000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9f9a000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9f99000
arch_prctl(ARCH_SET_FS, 0x7f1ed9f9a700) = 0
mprotect(0x3ebab89000, 16384, PROT_READ) = 0
mprotect(0x3eba21f000, 4096, PROT_READ) = 0
munmap(0x7f1ed9f9c000, 215683)          = 0
brk(0)                                  = 0x2238000
brk(0x2259000)                          = 0x2259000
open("/proc/devices", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9fd0000
read(3, "Character devices:\n  1 mem\n  4 /"..., 1024) = 617
close(3)                                = 0
munmap(0x7f1ed9fd0000, 4096)            = 0
open("/dev/sdb", O_RDONLY|O_NONBLOCK)   = 3
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1ed9fd0000
write(1, "VPD INQUIRY, page code=0xb2:\n", 29VPD INQUIRY, page code=0xb2:
) = 29
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 16), ...}) = 0
ioctl(3, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 01, b2, 00, fc, 00], mx_sb_len=32, iovec_count=0, dxfer_len=252, timeout=60000, flags=0

###
Syslog on host:

May  7 19:06:08 cdc-r710s3 abrt[4160]: Saved core dump of pid 7720 (/usr/libexec/qemu-kvm) to /var/spool/abrt/ccpp-2013-05-07-19:05:16-7720 (8961892352 bytes)
May  7 19:06:08 cdc-r710s3 abrtd: Directory 'ccpp-2013-05-07-19:05:16-7720' creation detected
May  7 19:06:08 cdc-r710s3 abrt[4160]: /var/spool/abrt is 26901360218 bytes (more than 1279MiB), deleting 'ccpp-2013-05-04-01:52:18-2773'
May  7 19:06:08 cdc-r710s3 kernel: br0: port 2(vnet0) entering disabled state
May  7 19:06:08 cdc-r710s3 kernel: device vnet0 left promiscuous mode
May  7 19:06:08 cdc-r710s3 kernel: br0: port 2(vnet0) entering disabled state
May  7 19:06:08 cdc-r710s3 kernel: br1: port 2(vnet1) entering disabled state
May  7 19:06:08 cdc-r710s3 kernel: device vnet1 left promiscuous mode
May  7 19:06:08 cdc-r710s3 kernel: br1: port 2(vnet1) entering disabled state
May  7 19:06:08 cdc-r710s3 kernel: br2: port 2(vnet2) entering disabled state
May  7 19:06:08 cdc-r710s3 kernel: device vnet2 left promiscuous mode
May  7 19:06:08 cdc-r710s3 kernel: br2: port 2(vnet2) entering disabled state
####

I will let you know where you can get the core file with email.

Thanks,
Richard.
Comment 24 Sibiao Luo 2013-07-03 03:05:19 UTC 
Hi all,

   We buy a HITACHI storage(HGST: Touro Mobile 3.0 0) and have tested it that did not hit this issue, it will prompt that page not supported.

host info:
kernel-2.6.32-392.el6.x86_64
qemu-kvm-0.12.1.2-2.376.el6.x86_64
guest info:
kernel-2.6.32-392.el6.x86_64

# /usr/libexec/qemu-kvm -S -M rhel6.4.0 -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -name sluo-test -uuid `uuidgen` -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/RHEL-Server-6.4-64-virtio.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-scsi-pci,bus=pci.0,addr=0x3,id=scsi0 -device scsi-hd,bus=scsi0.0,drive=drive-system-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2e:5f:0a:1d:b1,bus=pci.0,addr=0x4,bootindex=2,ioeventfd=off -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x5 -qmp tcp:0:4444,server,nowait -k en-us -boot menu=on -vnc :2 -spice disable-ticketing,port=5932 -vga qxl -monitor stdio
(qemu) device_add virtio-scsi-pci,bus=pci.0,addr=0x6,id=scsi1
(qemu) __com.redhat_drive_add file=/dev/disk/by-path/pci-0000:00:1d.0-usb-0:1.7:1.0-scsi-0:0:0:0,id=drive-data-disk,format=raw,cache=none
(qemu) device_add scsi-block,bus=scsi1.0,drive=drive-data-disk,id=data-disk
(qemu) info block
drive-system-disk: removable=0 io-status=ok file=/home/RHEL-Server-6.4-64-virtio.qcow2 ro=0 drv=qcow2 encrypted=0
ide1-cd0: removable=1 locked=0 tray-open=0 io-status=ok [not inserted]
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]
drive-data-disk: removable=0 io-status=ok file=/dev/disk/by-path/pci-0000:00:1d.0-usb-0:1.7:1.0-scsi-0:0:0:0 ro=0 drv=raw encrypted=0

# lsscsi
[1:0:0:0]    cd/dvd  QEMU     QEMU DVD-ROM     0.12  /dev/sr0 
[2:0:0:0]    disk    QEMU     QEMU HARDDISK    0.12  /dev/sda 
[3:0:0:0]    disk    HGST     Touro Mobile 3.0 0     /dev/sdb 

# sg_inq -e /dev/sdb
VPD INQUIRY, page code=0x00:
   [PQual=0  Peripheral device type: disk]
   Supported VPD pages:
     0x0	Supported VPD pages
     0x80	Unit serial number
     0x83	Device identification
     0xb0	Block limits (sbc2)

# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)
# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)
# sg_inq -H -p 0x01 /dev/sdb
VPD INQUIRY, page code=0x01:
    inquiry: field in cdb illegal (page not supported)

# sg_inq -H -p 0x80 /dev/sdb
VPD INQUIRY, page code=0x80:
 00     00 80 00 14 32 33 30 30  30 31 32 30 30 30 34 30    ....230001200040
 10     35 30 33 31 30 30 31 32                             50310012
# sg_inq -H -p 0x83 /dev/sdb
VPD INQUIRY, page code=0x83:
 00     00 83 00 0c 01 03 00 08  50 00 00 00 00 00 00 01    ........P.......
# sg_inq -H -p 0xb0 /dev/sdb
VPD INQUIRY, page code=0xb0:
 00     00 b0 00 3c 00 00 00 08  00 00 ff ff 00 00 ff ff    ...<............
 10     00 00 ff ff 00 00 00 00  00 00 00 00 00 00 00 00    ................
 20     00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00    ................
 30     00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00    ................


Best Regards,
sluo
Comment 25 Sibiao Luo 2013-07-03 05:35:10 UTC 
Hi kenny_sun,

    Could you help see comment #24? I did not reproduce this issue, could you help me check if our storage is different from yours or other reasons? 

Best Regards,
sluo
Comment 26 Kenny Sun 2013-07-03 07:28:19 UTC 
Hi Sibiao,

We hit the issue with Hitachi USPVM. Not sure if the array difference led to different results.

Did you get a chance to analyse the core file? Or do you need further input from us to help RCA the problem?

Thanks,
Kenny
Comment 30 Paolo Bonzini 2013-10-08 11:23:30 UTC 
It could be a one line fix:

   assert(req->sense_len < sizeof(req->sense));

must be

   assert(req->sense_len <= sizeof(req->sense));

I'll provide a test build later today.
Comment 31 Ademar Reis 2013-10-08 14:09:53 UTC 
(In reply to Paolo Bonzini from comment #30)
> It could be a one line fix:
> 
>    assert(req->sense_len < sizeof(req->sense));
> 
> must be
> 
>    assert(req->sense_len <= sizeof(req->sense));
> 
> I'll provide a test build later today.

So I guess it's going to be tested for sanity only (no reproducer, but the code is wrong and you'll fix it).
Comment 32 Paolo Bonzini 2013-10-08 14:59:48 UTC 
Yes, either sanity only or OtherQA.
Comment 34 Ademar Reis 2013-10-09 13:14:11 UTC 
Kenny: would you be able to run some tests on an updated package, to check if indeed fixes the bug on your environment? We've found what appears to be the root cause of the problem, but we're having trouble reproducing the crash on our side.
Comment 35 Paolo Bonzini 2013-10-10 13:37:51 UTC 
Testing the scsi-block device is good for sanity checking the patch.
Comment 36 Qunfang Zhang 2013-10-11 02:37:39 UTC 
Thanks for the confirmation, then QE will run a round of scsi-block function test with P1 cases for sanity. And adding the neeinfo to Kenny back according to Ademar's comment 34.
Comment 37 Kenny Sun 2013-10-11 02:41:33 UTC 
Thanks Ademar/Paolo! Which package I should upgrade to try reproducing the issue?
Comment 38 Qunfang Zhang 2013-10-15 06:37:55 UTC 
Hi, Paolo

Is there an update package to provide to Kenny to have to try?  Thanks.
Comment 39 Miroslav Rezanina 2013-10-15 09:34:19 UTC 
Fix committed in qemu-kvm-0.12.1.2-2.413.el6
Comment 40 Paolo Bonzini 2013-10-15 12:17:53 UTC 
I placed test packages at http://people.redhat.com/pbonzini/bz956929
Comment 41 Kenny Sun 2013-10-16 02:29:19 UTC 
Thanks Paolo and all.

I will arrange a verification shortly with the fix.
Comment 43 Qunfang Zhang 2013-10-18 10:26:57 UTC 
(In reply to Kenny Sun from comment #41)
> Thanks Paolo and all.
> 
> I will arrange a verification shortly with the fix.

Hi, Kenny

Is there some news about the verification? 

Many thanks,
Qunfang
Comment 44 Kenny Sun 2013-10-18 10:47:05 UTC 
Created attachment 813712 [details]
Verification logs 2013-10-18

Hi Qunfang/Paolo,

I have verified the fix on my test bed. The issue is not reproducible with the provided packages.

I also noticed that the new qemu packages depend on the following RPMs:
glusterfs-3.4.0.33rhs-1.el6rhs.x86_64.rpm      glusterfs-libs-3.4.0.33rhs-1.el6rhs.x86_64.rpm  spice-server-0.12.4-4.el6.x86_64.rpm
glusterfs-api-3.4.0.33rhs-1.el6rhs.x86_64.rpm

And the dependency was not there for the packages delivered with RHEL6U4. Can you confirm the change?

Verification logs attached.


Thanks,
Kenny
Comment 45 Qunfang Zhang 2013-10-21 03:48:21 UTC 
Hi, Kenny

Thanks a lot for the quick response and glad to know the issue is not reproducible with the provided packages. And for the dependency packages, I think they should be related to glusterfs and spice features, should not impact this bug even we did not update that. 

Hi, Paolo
Could you confirm? And please fix me if I was wrong. :)

Thanks,
Qunfang
Comment 47 Qunfang Zhang 2013-10-24 07:08:55 UTC 
Hi, Paolo

Could you double confirm comment 44?  We would like to verify this bug according to comment 44 and comment 46. Please correct me if something is wrong.

Thanks,
Qunfang
Comment 48 Paolo Bonzini 2013-10-24 23:22:42 UTC 
Yes, the bug can be verified.  Thanks!
Comment 50 errata-xmlrpc 2013-11-21 06:54:29 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1553.html
Note You need to log in before you can comment on or make changes to this bug.