Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1050824

Summary: qemu-kvm fail to generate EIO upon iscsi connection failure
Product: Red Hat Enterprise Linux 7 Reporter: Xu Han <xuhan>
Component: libiscsiAssignee: Paolo Bonzini <pbonzini>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: areis, bjoernv, hhuang, juzhang, knoel, pbonzini, rbalakri, virt-maint, xfu, xiagao
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-18 13:21:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dmesg log none

Description Xu Han 2014-01-09 07:15:41 UTC
Description of problem:
qemu-kvm fail to general EIO when met iscsi connection failure.

Version-Release number of selected component (if applicable):
qemu-kvm-1.5.3-31.el7.x86_64
kernel-debug-3.10.0-65.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. boot guest with a disk using iscsi storage.
# /usr/libexec/qemu-kvm -nodefaults -uuid 2de6c93b-a387-42e6-9810-c3a2a89b9242 -name ahci-test -M q35 -m 4G -cpu SandyBridge -smp 4,threads=2,cores=2,sockets=1 -qmp tcp:0:5550,server,nowait -rtc base=utc,clock=host,driftfix=slew -vga qxl -boot order=c,menu=off -spice disable-ticketing,port=5930 -device ahci,id=ahci0 -drive file=iscsi://10.66.6.145:3260/iqn.2001-04.com.example:disk/1,if=none,id=guest-img,cache=none,aio=native -device ide-hd,drive=guest-img,id=os-disk,bus=ahci0.0,unit=0 -netdev tap,id=tap0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=tap0,mac=02:e0:e1:e2:e3:ec,id=net0 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 \
-device ahci,id=ahci1 \
-drive file=iscsi://10.66.6.145:3260/iqn.2001-04.com.example:resize/1,if=none,id=disk1,cache=none,aio=native,rerror=stop,werror=stop \
-device ide-hd,drive=disk1,id=data-disk,bus=ahci1.0,unit=0

2. do some I/O operation in guest.
# dd if=/dev/zero of=/mnt/lv1/ddfile

3. use iptable reject iscsi service connection in host.
# iptables -A OUTPUT -p tcp -d 10.66.6.145 --dport 3260 -j DROP
# iptables -A INPUT -p tcp -d 10.66.6.145 --dport 3260 -j DROP

Actual results:
qemu-kvm: iSCSI: NOP timeout. Reconnecting...

Expected results:
qemu-kvm could general EIO message like:
block I/O error in device 'data-disk': Input/output error (5)

Additional info:
after resume iscsi service connection, would see cpu stuck and ata failure in guest dmesg log.
BUG: soft lockup - CPU#0 stuck for 46s! [kworker/0:1:43]
[ 1119.873080] Code: c3 29 5b 00 30 86 4e 81 c7 05 01 2a 5b 00 78 00 00 00 5d c3 0f 1f 80 00 00 00 00 66 66 66 66 90 8b 15 61 66 60 00 55 48 89 e5 ed <89> c6 81 e6 ff ff ff 00 ed 89 c1 81 e1 ff ff ff 00 ed 25 ff ff 
[ 1129.874322] NMI backtrace for cpu 1
[ 1129.874322] CPU: 1 PID: 250 Comm: scsi_eh_0 Not tainted 3.10.0-65.el7.x86_64.debug #1
[ 1129.874322] Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
[ 1129.874322] task: ffff880145150000 ti: ffff880144722000 task.ti: ffff880144722000
[ 1129.874322] RIP: 0010:[<ffffffffa00e526a>] 
[ 1129.874322]  [<ffffffffa00e526a>] ahci_freeze+0x2a/0x30 [libahci]
[ 1129.874322] RSP: 0018:ffff880144723d20  EFLAGS: 00010046
[ 1129.874322] RAX: 0000000000000000 RBX: ffff880144728000 RCX: 000000000001000b
[ 1129.874322] RDX: ffffc9000067c100 RSI: ffff88014472bc98 RDI: ffff880144728000
[ 1129.874322] RBP: ffff880144723d20 R08: 0000000000000000 R09: 0000000000000001
[ 1129.874322] R10: fffffffff66c0265 R11: 0000000000000000 R12: ffff880144728000
[ 1129.874322] R13: ffff8801447281b8 R14: 000000000000001f R15: ffff880144723d78
[ 1129.874322] FS:  0000000000000000(0000) GS:ffff88014b000000(0000) knlGS:0000000000000000
[ 1129.874322] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1129.874322] CR2: 00007f5499ecd000 CR3: 0000000142863000 CR4: 00000000000406e0
[ 1129.874322] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1129.874322] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1129.874322] Stack:
[ 1129.874322]  ffff880144723d38
[ 1129.874322]  ffffffffa008e669
[ 1129.874322]  ffff880144723d90
[ 1129.874322]  ffff880144723d80
[ 1129.874322]  ffffffffa008e81d
[ 1129.874322]  0000000000000296
[ 1129.874322]  ffff88014472bc98
[ 1129.874322]  ffff88014502d000
[ 1129.874322]  ffff880144723d90
[ 1129.874322]  ffff880144728000
[ 1129.874322]  0000000000000000
[ 1129.874322]  0000000000000000
[ 1129.874322] Call Trace:
[ 1129.874322]  [<ffffffffa008e669>] __ata_port_freeze+0x29/0x50 [libata]
[ 1129.874322]  [<ffffffffa008e81d>] ata_scsi_cmd_error_handler+0x10d/0x160 [libata]
[ 1129.874322]  [<ffffffffa00942cc>] ata_scsi_error+0x8c/0xd0 [libata]
[ 1129.874322]  [<ffffffff81437f23>] scsi_error_handler+0x123/0x7f0
[ 1129.874322]  [<ffffffff81437e00>] ? scsi_eh_get_sense+0x2a0/0x2a0
[ 1129.874322]  [<ffffffff81096efd>] kthread+0xed/0x100
[ 1129.874322]  [<ffffffff81096e10>] ? insert_kthread_work+0x80/0x80
[ 1129.874322]  [<ffffffff8168a86c>] ret_from_fork+0x7c/0xb0
[ 1129.874322]  [<ffffffff81096e10>] ? insert_kthread_work+0x80/0x80
[ 1129.874322] Code: 
[ 1129.874322] 90 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 90 
[ 1129.874322] 48 
[ 1129.874322] 8b 
[ 1129.874322] 87 
[ 1129.874322] 30 
[ 1129.874322] 36 
[ 1129.874322] 00 
[ 1129.874322] 00 
[ 1129.874322] 55 
[ 1129.874322] 48 
[ 1129.874322] 89 
[ 1129.874322] e5 
[ 1129.874322] 48 
[ 1129.874322] 8b 
[ 1129.874322] 50 
[ 1129.874322] 60 
[ 1129.874322] 8b 
[ 1129.874322] 47 
[ 1129.874322] 2c 
[ 1129.874322] 48 
[ 1129.874322] 8b 
[ 1129.874322] 12 
[ 1129.874322] c1 
[ 1129.874322] e0 
[ 1129.874322] 07 
[ 1129.874322] 48 
[ 1129.874322] 8d 
[ 1129.874322] 94 
[ 1129.874322] 02 
[ 1129.874322] 00 
[ 1129.874322] 01 
[ 1129.874322] 00 
[ 1129.874322] 00 
[ 1129.874322] 31 
[ 1129.874322] c0 
[ 1129.874322] 89 
[ 1129.874322] 42 
[ 1129.874322] 14 
[ 1129.874322] <5d> 
[ 1129.874322] c3 
[ 1129.874322] 0f 
[ 1129.874322] 1f 
[ 1129.874322] 40 
[ 1129.874322] 00 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 66 
[ 1129.874322] 90 
[ 1129.874322] 48 
[ 1129.874322] 8b 
[ 1129.874322] 87 
[ 1129.874322] 30 
[ 1129.874322] 36 
[ 1129.874322] 00 
[ 1129.874322] 00 
[ 1129.874322] 55 
[ 1129.874322] 4c 
[ 1129.874322] 8b 
...
[ 1180.011866] ata1.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x6 frozen
[ 1180.011868] ata1.00: failed command: WRITE FPDMA QUEUED
[ 1180.011872] ata1.00: cmd 61/00:00:d8:28:a4/01:00:03:00:00/40 tag 0 ncq 131072 out
         res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[ 1180.011875] ata1.00: status: { DRDY }
[ 1180.011876] ata1.00: failed command: WRITE FPDMA QUEUED
[ 1180.011879] ata1.00: cmd 61/00:08:d8:29:a4/01:00:03:00:00/40 tag 1 ncq 131072 out
         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
...

Comment 2 Xu Han 2014-01-09 07:27:03 UTC
Created attachment 847503 [details]
dmesg log

Comment 4 xiagao 2014-11-06 11:15:15 UTC
 
after step 3,clear the iptables in host.
# iptables -D OUTPUT -p tcp -d 10.66.6.145 --dport 3260 -j DROP
# iptables -D INPUT -p tcp -d 10.66.6.145 --dport 3260 -j DROP

the actual result in qemu-kvm:
 block I/O error in device 'scsi_drive': Input/output error (5)
 block I/O error in device 'scsi_drive': Input/output error (5)
 block I/O error in device 'scsi_drive': Input/output error (5)

Comment 6 Paolo Bonzini 2015-09-18 13:21:37 UTC

*** This bug has been marked as a duplicate of bug 1229254 ***