Bug 867401

Summary: wrong I/O error prompts and fail to resume VM when generate EIO firewalling the iSCSI port with I/O in guest
Product: Red Hat Enterprise Linux 6 Reporter: Sibiao Luo <sluo>
Component: qemu-kvmAssignee: Kevin Wolf <kwolf>
Status: CLOSED NEXTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.4CC: acathrow, areis, bsarathy, chayang, dyasny, flang, juzhang, michen, mkenneth, pbonzini, qzhang, sluo, virt-maint
Target Milestone: rc   
Target Release: 6.5   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-06-05 19:31:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sibiao Luo 2012-10-17 12:53:26 UTC 
Description of problem:
boot a guest with a data disk on iSCSI storage, and do I/O to the data disk, then firewall the iSCSI port with iptables and generate EIO with wrong I/O error prompts. then stop the iptables and resume the VM via 'cont' in QEMU monitor, it fail to resume the VM.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm
2.6.32-331.el6.x86_64
qemu-kvm-0.12.1.2-2.327.el6.x86_64
guest info:
# uname -r
2.6.32-331.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1.prepare a guest image/data disk and iSCSI server.
# qemu-img create -f qcow2 /mnt/my-data-disk.qcow2 10G
# iscsiadm --mode discovery --type sendtargets --portal 10.66.6.78 –discover
# iscsiadm -m node -T test4 -p 10.66.6.78 -l
Logging in to [iface: default, target: test4, portal: 10.66.6.78,3260] (multiple)

Login to [iface: default, target: test4, portal: 10.66.6.78,3260] successful.
...
2.boot a guest with a data disk on iSCSI storage.
eg: # /usr/libexec/qemu-kvm -M rhel6.4.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_migration -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/mnt/RHEL-Server-6.3-64-sluo.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on,fd=6 6<>/dev/tap6 -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2E:5F:0A:0D:B1,bus=pci.0,addr=0x5 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5931,disable-ticketing,seamless-migration=off -vga qxl -global qxl-vga.vram_size=67108864 -device intel-hda,id=sound0,bus=pci.0,addr=0x6 -device hda-duplex -device usb-ehci,id=ehci,addr=0x7 -chardev spicevmc,name=usbredir,id=usbredirchardev1 -device usb-redir,chardev=usbredirchardev1,id=usbredirdev1,bus=ehci.0,debug=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -nodefaults -serial unix:/tmp/ttyS0,server,nowait -qmp tcp:0:4444,server,nowait -drive file=/mnt/my-data-disk.qcow2,if=none,id=data-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,drive=data-disk,id=sluo-disk,bus=pci.0,addr=0x9 -boot menu=on -monitor stdio
3.mkfs.ext4 to the data disk and mount it to /mnt.
# mkfs.ext4 /dev/vdb
# mount /dev/vdb /mnt/
4.do dd in loop.
# while true; do dd if=/dev/zero of=/mnt/sluo bs=1M count=100; done
5.firewall the iSCSI port in host.
# iptables -A INPUT -p tcp -s $ip_addr --dport $port -j DROP
6.stop the iptables and resume the VM via 'cont' in QEMU monitor.
# service iptables stop
(QEMU) cont

Actual results:
after the step5, generate EIO as following:
(qemu) block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
...
after the step6,
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)
block I/O error in device 'data-disk': No space left on device (28)


Expected results:
after the step5, generate EIO as following:
(qemu) block I/O error in device 'data-disk': Input/output error (5)
block I/O error in device 'data-disk': Input/output error (5)
block I/O error in device 'data-disk': Input/output error (5)
...
after the step6, it should resume the VM successfully.

Additional info:
after the step6, it will core dump, i see this is relate to the qxl, so i will open a new bug for tracing.
Comment 2 Sibiao Luo 2012-10-17 13:04:09 UTC 
(In reply to comment #0)
> Description of problem:
> boot a guest with a data disk on iSCSI storage, and do I/O to the data disk,
> then firewall the iSCSI port with iptables and generate EIO with wrong I/O
> error prompts. then stop the iptables and resume the VM via 'cont' in QEMU
> monitor, it fail to resume the VM.
> 
> 
> Additional info:
> after the step6, it will core dump, i see this is relate to the qxl, so i
> will open a new bug for tracing.
see bug 867405

cpu info:
processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping	: 7
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6784.19
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 3 Ademar Reis 2012-10-19 21:43:17 UTC 
So, bug 867405 is about the coredump (qxl related?) and this one is about the error messages and failure to continue.
Comment 4 Sibiao Luo 2012-10-20 02:27:53 UTC 
(In reply to comment #3)
> So, bug 867405 is about the coredump (qxl related?) and this one is about
> the error messages and failure to continue.
Yes, you are correct.
Comment 5 Kevin Wolf 2012-10-22 08:52:29 UTC 
Can you run strace on the qemu process and check what return values it gets from the kernel for it's preadv/pwritev syscalls?
Comment 6 Sibiao Luo 2012-10-23 04:11:22 UTC 
(In reply to comment #5)
> Can you run strace on the qemu process and check what return values it gets
> from the kernel for it's preadv/pwritev syscalls?

kwolf, 

   i didnot see any preadv/pwrite syscalls during I/O to the data disk, no matter firewall the iSCSI port or not. It's possible that strace in RHEL 6 doesn't know the preadv/pwritev syscalls yet. Please let me know if you need any info. 
# grep preadv --color /home/strace.log 
     <-- got nothing
# grep pwritev --color /home/strace.log
     <-- got nothing

I paste some about read/write syscalls during the I/O as following,
...
select(43, [0 5 8 12 14 15 18 25 27 29 30 31 32 33 34 37 42], [], [], {1, 0}) = 1 (in [29], left {0, 999977})
read(29, "\16\0\0\0\0\0\0\0\376\377\377\377\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 128) = 128
rt_sigaction(SIGALRM, NULL, {0x7faadbcc2680, ~[KILL STOP RTMIN RT_1], SA_RESTORER, 0x7faadb625500}, 8) = 0
write(9, "\0", 1)                       = 1
write(28, "\1\0\0\0\0\0\0\0", 8)        = 8
)        = 8
read(29, 0x7fff8b3b4f80, 128)           = -1 EAGAIN (Resource temporarily unavailable)
timer_gettime(0, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0, 0, {it_interval={0, 0}, it_value={0, 2000000}}, NULL) = 0
NULL) = 0
select(43, [0 5 8 12 14 15 18 25 27 29 30 31 32 33 34 37 42], [], [], {1, 0}) = 2 (in [8 27], left {0, 999996})
read(27, "\1\0\0\0\0\0\0\0", 4096)      = 8
read(27, 0x7fff8b3b4010, 4096)          = -1 EAGAIN (Resource temporarily unavailable)
read(8, "\0", 512)                      = 1
read(8, 0x7fff8b3b4e10, 512)            = -1 EAGAIN (Resource temporarily unavailable)
select(43, [0 5 8 12 14 15 18 25 27 29 30 31 32 33 34 37 42], [], [], {1, 0}) = 1 (in [29], left {0, 998204})
...

Best Regards.
sluo
Comment 7 langfang 2012-10-24 10:05:58 UTC 
test this case hit the follow problem
version:
host(AMD)
# uname -r
2.6.32-327.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.329.el6.x86_64
# rpm -q seabios
seabios-0.6.1.2-25.el6.x86_64
guest:win2008r2

the steps 1-4 as same as the description
step 5.
on host:
#service iscsi stop

results:
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
[Thread 0x7ffff005f700 (LWP 6784) exited]
[Thread 0x7ffece9fb700 (LWP 6783) exited]
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
block I/O error in device 'drive-virtio-disk1': Input/output error (5)
(/usr/bin/gdb:6688): Spice-Warning **: red_parse_qxl.c:406:red_get_image: guest error: missing palette on bitmap format=2

(/usr/bin/gdb:6688): Spice-Warning **: red_parse_qxl.c:406:red_get_image: guest error: missing palette on bitmap format=2


Message from syslogd@localhost at Oct 24 17:44:07 ...
 kernel:BUG: soft lockup - CPU#2 stuck for 67s! [scsi_wq_6:6604]

Message from syslogd@localhost at Oct 24 17:45:31 ...
 kernel:BUG: soft lockup - CPU#2 stuck for 67s! [scsi_wq_6:6604]

Message from syslogd@localhost at Oct 24 17:46:55 ...
 kernel:BUG: soft lockup - CPU#2 stuck for 68s! [scsi_wq_6:6604]

Message from syslogd@localhost at Oct 24 17:48:19 ...
 kernel:BUG: soft lockup - CPU#2 stuck for 67s! [scsi_wq_6:6604]
Comment 8 Kevin Wolf 2012-10-24 10:18:27 UTC 
(In reply to comment #6)
>    i didnot see any preadv/pwrite syscalls during I/O to the data disk, no
> matter firewall the iSCSI port or not. It's possible that strace in RHEL 6
> doesn't know the preadv/pwritev syscalls yet. Please let me know if you need
> any info.

I looked into the code and it can be non-vectored pread/pwrite syscalls as well. Do you see these?
Comment 9 Sibiao Luo 2012-11-05 13:07:33 UTC 
(In reply to comment #0)
> Description of problem:
> boot a guest with a data disk on iSCSI storage, and do I/O to the data disk,
> then firewall the iSCSI port with iptables and generate EIO with wrong I/O
> error prompts. then stop the iptables and resume the VM via 'cont' in QEMU
> monitor, it fail to resume the VM.
> 
> Actual results:
> after the step5, generate EIO as following:
> (qemu) block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> ...
> after the step6,
> (qemu) info status 
> VM status: paused (io-error)
> (qemu) cont
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> block I/O error in device 'data-disk': No space left on device (28)
> 
I do many times, did not meet this 'No space left on device' prompt. I remember i create 20G block device and make file system to it, then mount it to /mnt, the system image and data disk locate to the /mnt, and i do I/O just 100M, maybe there are many data located in /mnt, or the system image should not locate in /mnt.

------------------------------------------------------------------------------

This time i use the system image locate in local host, and with the data disk located on the 'file: iSCSI' storage. But i meet anther prompts: 'Input/output error (5)' and 'Read-only file system (30)', i donot know whether is expected.
BTW, if use the iscsi block devive, it can resume the guest but call trace, you could refer to bug 873246.
(qemu) info block
drive-virtio-disk0: removable=0 io-status=ok file=/home/RHEL-Server-6.3-64-copy.qcow2 ro=0 drv=qcow2 encrypted=0
data-disk: removable=0 io-status=failed file=/mnt/my-data-disk.qcow2 ro=0 drv=qcow2 encrypted=0
(qemu) info status 
VM status: running

General EIO and the VM is in paused status after firewall the iSCSI port.
- in host:
# iptables -A OUTPUT -p tcp -d 10.66.90.0/24 --dport 3260 -j DROP
- in qemu:
(qemu) block I/O error in device 'data-disk': Input/output error (5)
block I/O error in device 'data-disk': Input/output error (5)

When i stop the firewall but fail to resume the VM.
- in host:     
# service iptables stop
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:                               [  OK  ]
# netstat -ant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
...
tcp        0      0 10.66.9.242:52658           10.66.90.109:3260           ESTABLISHED
...
- in qemu:
(qemu) info status 
VM status: paused (io-error)
(qemu) 
(qemu) cont
(qemu) block I/O error in device 'data-disk': Read-only file system (30)
block I/O error in device 'data-disk': Input/output error (5)


Actual results:
1.should give the right prompt.
2.can resume the VM successfully and the guest works well.


Additional info:
host info:
# uname -r && rpm -q qemu-kvm
2.6.32-335.el6.x86_64
qemu-kvm-0.12.1.2-2.331.el6.x86_64
guest info:
kernel-2.6.32-336.el6.x86_64

Qemu-kvm command line:
# /usr/libexec/qemu-kvm -M rhel6.4.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_test -uuid 7818b04d-aa83-4fb5-8ae5-e6024ebf6299 -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/RHEL-Server-6.3-64-copy.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2E:5F:0A:0D:B1,bus=pci.0,addr=0x4 -spice port=5931,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -device intel-hda,id=sound0,bus=pci.0,addr=0x5 -device hda-duplex -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -drive file=/mnt/my-data-disk.qcow2,if=none,id=data-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=data-disk,id=sluo-disk -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -nodefaults -serial unix:/tmp/ttyS0,server,nowait -boot menu=on -monitor stdio
Comment 10 Sibiao Luo 2012-11-05 14:31:44 UTC 
(In reply to comment #9)
> Actual results:
> 1.should give the right prompt.
> 2.can resume the VM successfully and the guest works well.
> 
it's wrong, this should be the expected resulets.

Best Regards.
sluo
Comment 11 Sibiao Luo 2012-11-09 10:53:54 UTC 
I retest it again and got all the strace log, i will attach it later. btw, i did not see any non-vectored pread/pwrite and preadv/pwritev syscalls from strace log.

host info:
kernel-2.6.32-339.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.334.el6.x86_64
seabios-0.6.1.2-25.el6.x86_64
guest info:
kernel-2.6.32-339.el6.x86_64

make the data disk located on the 'file: iSCSI' storage and mount it to /mnt.
# lvcreate -L 30G -n lv-90.100-data-disk-test vg-90.100-sluo
# mkfs.ext4 /dev/vg-90.100-sluo/lv-90.100-data-disk-test
# mount /dev/vg-90.100-sluo/lv-90.100-data-disk-test /mnt/
# qemu-img create -f qcow2 /mnt/my-data-disk.qcow2 5G

# strace -o /home/bug867401-strace-log.txt /usr/libexec/qemu-kvm -M rhel6.4.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_test -uuid 7818b04d-aa83-4fb5-8ae5-e6024ebf6299 -rtc base=localtime,clock=host,driftfix=slew -drive file=/mnt/rhel6.4-64bit.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=08:2E:5F:0A:0D:B1,bus=pci.0,addr=0x4 -spice port=5931,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -device intel-hda,id=sound0,bus=pci.0,addr=0x5 -device hda-duplex -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -drive file=/mnt/my-data-disk.qcow2,if=none,id=data-disk,format=qcow2,cache=none,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=data-disk,id=sluo-disk -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -nodefaults -serial unix:/tmp/ttyS0,server,nowait -boot menu=on -monitor stdio

after i firewall the iSCSI port in host, it will general EIO.
(qemu) info status 
VM status: running
(qemu) block I/O error in device 'data-disk': Input/output error (5)
block I/O error in device 'drive-virtio-disk0': Input/output error (5)
block I/O error in device 'drive-virtio-disk0': Read-only file system (30)
block I/O error in device 'drive-virtio-disk0': Read-only file system (30)
(qemu) info status 
VM status: paused (io-error)

stop the iptables but fail to resume the VM via 'cont' in QEMU monitor, i tried many times and wait a long time, it fail to resume.
# service iptables stop
(qemu) cont
(qemu) block I/O error in device 'data-disk': Read-only file system (30)
block I/O error in device 'drive-virtio-disk0': Read-only file system (30)
block I/O error in device 'drive-virtio-disk0': Read-only file system (30)
Comment 13 Kevin Wolf 2012-11-27 15:11:20 UTC 
Please try the -f option to strace, so that the threads are included in the trace. qemu doesn't perform I/O requests from the main thread.
Comment 15 RHEL Program Management 2012-12-14 07:10:26 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 20 juzhang 2013-05-20 07:37:41 UTC 
Hi Sluo,

Please re-test this issue if the internal build which is based qemu1.5 comes out.
Comment 21 Sibiao Luo 2013-05-30 08:16:12 UTC 
(In reply to juzhang from comment #20)
> Hi Sluo,
> 
> Please re-test this issue if the internal build which is based qemu1.5 comes
> out.

Test it on qemu-kvm-1.5.0-2.el7.x86_64 with comment#0 steps, but met different results that cann't general any EIO at all.

host info:
kernel-3.10.0-0.rc2.57.el7.x86_64
qemu-kvm-1.5.0-2.el7.x86_64
guest info:
kernel-3.10.0-0.rc2.57.el7.x86_64

- I.If use a lvm as data disk.
eg:...-drive file=/dev/sluo_vg/my-data-disk.raw,if=none,id=drive-data-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk
1.after firewall the iSCSI port, the VM was in paused status but did not general EIO.
(qemu) info status 
VM status: running
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)
2.after stop the iptables and resume the VM via 'cont' in QEMU monitor successfully.
(qemu) cont
(qemu) info status 
VM status: running
(qemu) info status 
VM status: running

- II.Create a data disk located on the iSCSI storage('file: iSCSI').
host]# mkfs.ext4 /dev/sluo_vg/my-data-disk.raw
host]# mount /dev/sluo_vg/my-data-disk.raw /mnt/
host]# qemu-img create -f qcow2 /mnt/my-data-disk.qcow2 10G
e.g:...-drive file=/mnt/my-data-disk.qcow2,if=none,id=drive-data-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk

1.after firewall the iSCSI port, the VM was in paused status but did not general EIO.
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)
2.after stop the iptables but fail to resume the VM via 'cont' in QEMU monitor.
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)
(qemu) cont
(qemu) info status 
VM status: paused (io-error)

got dmesg from host that flood with many error.
# dmesg 
[264917.755874]  session1: session recovery timed out after 120 secs
[264917.761876] sd 6:0:0:0: rejecting I/O to offline device
[264917.762644] sd 6:0:0:0: [sdb] killing request
[264917.762649] sd 6:0:0:0: rejecting I/O to offline device
[264917.763404] sd 6:0:0:0: [sdb] killing request
[264917.763407] sd 6:0:0:0: rejecting I/O to offline device
[264917.764160] sd 6:0:0:0: [sdb] killing request
[264917.764162] sd 6:0:0:0: rejecting I/O to offline device
[264917.764917] sd 6:0:0:0: [sdb] killing request
[264917.764919] sd 6:0:0:0: rejecting I/O to offline device
[264917.765672] sd 6:0:0:0: [sdb] killing request
[264917.765674] sd 6:0:0:0: rejecting I/O to offline device
[264917.766429] sd 6:0:0:0: [sdb] killing request
[264917.766431] sd 6:0:0:0: rejecting I/O to offline device
[264917.767185] sd 6:0:0:0: [sdb] killing request
[264917.767188] sd 6:0:0:0: rejecting I/O to offline device
[264917.767960] sd 6:0:0:0: [sdb] killing request
[264917.767962] sd 6:0:0:0: rejecting I/O to offline device
[264917.768767] sd 6:0:0:0: [sdb] killing request
[264917.768769] sd 6:0:0:0: rejecting I/O to offline device
[264917.769559] sd 6:0:0:0: [sdb] killing request
[264917.769561] sd 6:0:0:0: rejecting I/O to offline device
[264917.770302] sd 6:0:0:0: [sdb] killing request
[264917.770304] sd 6:0:0:0: rejecting I/O to offline device
[264917.771044] sd 6:0:0:0: [sdb] killing request
[264917.771046] sd 6:0:0:0: rejecting I/O to offline device
[264917.771803] sd 6:0:0:0: [sdb] killing request
[264917.771805] sd 6:0:0:0: rejecting I/O to offline device
[264917.772561] sd 6:0:0:0: [sdb] killing request
[264917.772563] sd 6:0:0:0: rejecting I/O to offline device
[264917.773319] sd 6:0:0:0: [sdb] killing request
[264917.773321] sd 6:0:0:0: rejecting I/O to offline device
[264917.774088] sd 6:0:0:0: [sdb] killing request
[264917.774090] sd 6:0:0:0: rejecting I/O to offline device
[264917.774845] sd 6:0:0:0: [sdb] killing request
[264917.774847] sd 6:0:0:0: rejecting I/O to offline device
[264917.775601] sd 6:0:0:0: [sdb] killing request
[264917.775603] sd 6:0:0:0: rejecting I/O to offline device
[264917.776356] sd 6:0:0:0: [sdb] killing request
[264917.776358] sd 6:0:0:0: rejecting I/O to offline device
[264917.777130] sd 6:0:0:0: [sdb] killing request
[264917.777132] sd 6:0:0:0: rejecting I/O to offline device
[264917.777886] sd 6:0:0:0: [sdb] killing request
[264917.777888] sd 6:0:0:0: rejecting I/O to offline device
[264917.778644] sd 6:0:0:0: [sdb] killing request
[264917.778646] sd 6:0:0:0: rejecting I/O to offline device
[264917.779400] sd 6:0:0:0: [sdb] killing request
[264917.779402] sd 6:0:0:0: rejecting I/O to offline device
[264917.780189] sd 6:0:0:0: [sdb] killing request
[264917.780191] sd 6:0:0:0: rejecting I/O to offline device
[264917.780946] sd 6:0:0:0: [sdb] killing request
[264917.780948] sd 6:0:0:0: rejecting I/O to offline device
[264917.781720] sd 6:0:0:0: [sdb] killing request
[264917.781722] sd 6:0:0:0: rejecting I/O to offline device
[264917.782490] sd 6:0:0:0: [sdb] killing request
[264917.782492] sd 6:0:0:0: rejecting I/O to offline device
[264917.783261] sd 6:0:0:0: [sdb] killing request
[264917.783263] sd 6:0:0:0: rejecting I/O to offline device
[264917.784017] sd 6:0:0:0: [sdb] killing request
[264917.784019] sd 6:0:0:0: rejecting I/O to offline device
[264917.784771] sd 6:0:0:0: [sdb] killing request
[264917.784773] sd 6:0:0:0: rejecting I/O to offline device
[264917.785526] sd 6:0:0:0: [sdb] killing request
[264917.785528] sd 6:0:0:0: rejecting I/O to offline device
[264917.786261] sd 6:0:0:0: [sdb] killing request
[264917.786289] sd 6:0:0:0: [sdb] Unhandled error code
[264917.786291] sd 6:0:0:0: [sdb]  
[264917.786294] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.786296] sd 6:0:0:0: [sdb] CDB: 
[264917.786298] Write(10): 2a 00 09 1b 9c 60 00 04 00 00
[264917.786307] blk_update_request: 23 callbacks suppressed
[264917.786309] end_request: I/O error, dev sdb, sector 152804448
[264917.787034] sd 6:0:0:0: rejecting I/O to offline device
[264917.787794] sd 6:0:0:0: [sdb] killing request
[264917.787797] sd 6:0:0:0: rejecting I/O to offline device
[264917.788555] sd 6:0:0:0: rejecting I/O to offline device
[264917.789309] sd 6:0:0:0: rejecting I/O to offline device
[264917.790060] sd 6:0:0:0: rejecting I/O to offline device
[264917.790819] sd 6:0:0:0: rejecting I/O to offline device
[264917.791608] sd 6:0:0:0: rejecting I/O to offline device
[264917.792381] sd 6:0:0:0: rejecting I/O to offline device
[264917.793135] sd 6:0:0:0: rejecting I/O to offline device
[264917.793870] sd 6:0:0:0: rejecting I/O to offline device
[264917.794569] sd 6:0:0:0: rejecting I/O to offline device
[264917.795256] sd 6:0:0:0: rejecting I/O to offline device
[264917.795891] sd 6:0:0:0: rejecting I/O to offline device
[264917.796508] sd 6:0:0:0: rejecting I/O to offline device
[264917.797102] sd 6:0:0:0: rejecting I/O to offline device
[264917.797665] sd 6:0:0:0: rejecting I/O to offline device
[264917.798204] sd 6:0:0:0: rejecting I/O to offline device
[264917.798722] sd 6:0:0:0: rejecting I/O to offline device
[264917.799219] sd 6:0:0:0: rejecting I/O to offline device
[264917.799675] sd 6:0:0:0: rejecting I/O to offline device
[264917.800112] sd 6:0:0:0: rejecting I/O to offline device
[264917.800491] sd 6:0:0:0: rejecting I/O to offline device
[264917.800887] sd 6:0:0:0: rejecting I/O to offline device
[264917.801304] sd 6:0:0:0: rejecting I/O to offline device
[264917.801685] sd 6:0:0:0: rejecting I/O to offline device
[264917.802079] sd 6:0:0:0: rejecting I/O to offline device
[264917.802456] sd 6:0:0:0: rejecting I/O to offline device
[264917.802857] sd 6:0:0:0: rejecting I/O to offline device
[264917.803246] sd 6:0:0:0: rejecting I/O to offline device
[264917.803631] sd 6:0:0:0: rejecting I/O to offline device
[264917.804014] sd 6:0:0:0: rejecting I/O to offline device
[264917.804382] sd 6:0:0:0: rejecting I/O to offline device
[264917.804726] sd 6:0:0:0: rejecting I/O to offline device
[264917.805082] sd 6:0:0:0: rejecting I/O to offline device
[264917.805437] sd 6:0:0:0: rejecting I/O to offline device
[264917.805779] sd 6:0:0:0: rejecting I/O to offline device
[264917.806120] sd 6:0:0:0: rejecting I/O to offline device
[264917.806465] sd 6:0:0:0: rejecting I/O to offline device
[264917.806820] sd 6:0:0:0: rejecting I/O to offline device
[264917.807190] sd 6:0:0:0: rejecting I/O to offline device
[264917.807546] sd 6:0:0:0: rejecting I/O to offline device
[264917.807899] sd 6:0:0:0: rejecting I/O to offline device
[264917.808241] sd 6:0:0:0: rejecting I/O to offline device
[264917.808594] sd 6:0:0:0: [sdb] Unhandled error code
[264917.808597] sd 6:0:0:0: [sdb]  
[264917.808599] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.808601] sd 6:0:0:0: [sdb] CDB: 
[264917.808602] Write(10): 2a 00 09 1b b0 60 00 04 00 00
[264917.808610] end_request: I/O error, dev sdb, sector 152809568
[264917.808611] Aborting journal on device dm-6-8.
[264917.808636] sd 6:0:0:0: rejecting I/O to offline device
[264917.808641] Buffer I/O error on device dm-6, logical block 1081344
[264917.808642] lost page write due to I/O error on dm-6
[264917.808646] JBD2: Error -5 detected when updating journal superblock for dm-6-8.
[264917.810647] sd 6:0:0:0: [sdb] Unhandled error code
[264917.810650] sd 6:0:0:0: [sdb]  
[264917.810651] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.810653] sd 6:0:0:0: [sdb] CDB: 
[264917.810654] Write(10): 2a 00 09 1b 80 60 00 04 00 00
[264917.810662] end_request: I/O error, dev sdb, sector 152797280
[264917.811194] sd 6:0:0:0: [sdb] Unhandled error code
[264917.811196] sd 6:0:0:0: [sdb]  
[264917.811198] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.811200] sd 6:0:0:0: [sdb] CDB: 
[264917.811201] Write(10): 2a 00 09 1b c4 70 00 04 00 00
[264917.811208] end_request: I/O error, dev sdb, sector 152814704
[264917.811711] sd 6:0:0:0: [sdb] Unhandled error code
[264917.811713] sd 6:0:0:0: [sdb]  
[264917.811715] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.811717] sd 6:0:0:0: [sdb] CDB: 
[264917.811718] Write(10): 2a 00 09 1b 88 60 00 04 00 00
[264917.811724] end_request: I/O error, dev sdb, sector 152799328
[264917.812244] sd 6:0:0:0: [sdb] Unhandled error code
[264917.812246] sd 6:0:0:0: [sdb]  
[264917.812248] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.812250] sd 6:0:0:0: [sdb] CDB: 
[264917.812251] Write(10): 2a 00 09 1b 77 c0 00 04 00 00
[264917.812258] end_request: I/O error, dev sdb, sector 152795072
[264917.812764] sd 6:0:0:0: [sdb] Unhandled error code
[264917.812766] sd 6:0:0:0: [sdb]  
[264917.812768] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.812770] sd 6:0:0:0: [sdb] CDB: 
[264917.812771] Write(10): 2a 00 09 1b 73 c0 00 04 00 00
[264917.812778] end_request: I/O error, dev sdb, sector 152794048
[264917.813126] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813127] sd 6:0:0:0: [sdb]  
[264917.813128] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813129] sd 6:0:0:0: [sdb] CDB: 
[264917.813129] Write(10): 2a 00 09 1b 67 c0 00 04 00 00
[264917.813133] end_request: I/O error, dev sdb, sector 152790976
[264917.813386] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813387] sd 6:0:0:0: [sdb]  
[264917.813388] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813389] sd 6:0:0:0: [sdb] CDB: 
[264917.813389] Write(10): 2a 00 09 1b 6f c0 00 04 00 00
[264917.813392] end_request: I/O error, dev sdb, sector 152793024
[264917.813647] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813648] sd 6:0:0:0: [sdb]  
[264917.813648] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813649] sd 6:0:0:0: [sdb] CDB: 
[264917.813650] Write(10): 2a 00 09 1b d8 70 00 04 00 00
[264917.813653] end_request: I/O error, dev sdb, sector 152819824
[264917.813912] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813913] sd 6:0:0:0: [sdb]  
[264917.813914] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813915] sd 6:0:0:0: [sdb] CDB: 
[264917.813915] Write(10): 2a 00 09 1b c0 70 00 04 00 00
[264917.813924] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813925] sd 6:0:0:0: [sdb]  
[264917.813925] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813926] sd 6:0:0:0: [sdb] CDB: 
[264917.813927] Write(10): 2a 00 09 1b 63 c0 00 04 00 00
[264917.813935] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813938] sd 6:0:0:0: [sdb]  
[264917.813941] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813944] sd 6:0:0:0: [sdb] CDB: 
[264917.813947] Write(10): 2a 00 09 1b 84 60 00 04 00 00
[264917.813957] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813958] sd 6:0:0:0: [sdb]  
[264917.813958] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813959] sd 6:0:0:0: [sdb] CDB: 
[264917.813960] Write(10): 2a 00 09 1b 5f c0 00 04 00 00
[264917.813968] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813969] sd 6:0:0:0: [sdb]  
[264917.813970] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813971] sd 6:0:0:0: [sdb] CDB: 
[264917.813971] Write(10): 2a 00 09 1b 90 60 00 04 00 00
[264917.813978] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813979] sd 6:0:0:0: [sdb]  
[264917.813980] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813981] sd 6:0:0:0: [sdb] CDB: 
[264917.813981] Write(10): 2a 00 09 1b d4 70 00 04 00 00
[264917.813988] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813989] sd 6:0:0:0: [sdb]  
[264917.813990] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.813991] sd 6:0:0:0: [sdb] CDB: 
[264917.813991] Write(10): 2a 00 09 1b 8c 60 00 04 00 00
[264917.813998] sd 6:0:0:0: [sdb] Unhandled error code
[264917.813999] sd 6:0:0:0: [sdb]  
[264917.814000] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814000] sd 6:0:0:0: [sdb] CDB: 
[264917.814001] Write(10): 2a 00 09 1b ac 60 00 04 00 00
[264917.814020] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814021] sd 6:0:0:0: [sdb]  
[264917.814021] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814022] sd 6:0:0:0: [sdb] CDB: 
[264917.814023] Write(10): 2a 00 09 1b 98 60 00 04 00 00
[264917.814030] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814031] sd 6:0:0:0: [sdb]  
[264917.814032] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814033] sd 6:0:0:0: [sdb] CDB: 
[264917.814034] Write(10): 2a 00 09 1b bc 60 00 00 10 00
[264917.814039] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814040] sd 6:0:0:0: [sdb]  
[264917.814040] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814041] sd 6:0:0:0: [sdb] CDB: 
[264917.814042] Write(10): 2a 00 09 1b b8 60 00 04 00 00
[264917.814050] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814051] sd 6:0:0:0: [sdb]  
[264917.814052] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814053] sd 6:0:0:0: [sdb] CDB: 
[264917.814053] Write(10): 2a 00 09 1b b4 60 00 04 00 00
[264917.814060] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814061] sd 6:0:0:0: [sdb]  
[264917.814062] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814063] sd 6:0:0:0: [sdb] CDB: 
[264917.814064] Write(10): 2a 00 09 1b bc 70 00 04 00 00
[264917.814084] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814085] sd 6:0:0:0: [sdb]  
[264917.814086] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814087] sd 6:0:0:0: [sdb] CDB: 
[264917.814088] Write(10): 2a 00 09 1b 6b c0 00 04 00 00
[264917.814096] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814097] sd 6:0:0:0: [sdb]  
[264917.814098] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814099] sd 6:0:0:0: [sdb] CDB: 
[264917.814099] Write(10): 2a 00 09 1b a4 60 00 04 00 00
[264917.814107] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814111] sd 6:0:0:0: [sdb]  
[264917.814114] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814116] sd 6:0:0:0: [sdb] CDB: 
[264917.814116] Write(10): 2a 00 09 1b d0 70 00 04 00 00
[264917.814124] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814125] sd 6:0:0:0: [sdb]  
[264917.814126] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814127] sd 6:0:0:0: [sdb] CDB: 
[264917.814127] Write(10): 2a 00 09 1b cc 70 00 04 00 00
[264917.814135] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814138] sd 6:0:0:0: [sdb]  
[264917.814141] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814142] sd 6:0:0:0: [sdb] CDB: 
[264917.814142] Write(10): 2a 00 09 1b a8 60 00 04 00 00
[264917.814149] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814150] sd 6:0:0:0: [sdb]  
[264917.814151] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814152] sd 6:0:0:0: [sdb] CDB: 
[264917.814152] Write(10): 2a 00 09 1b c8 70 00 04 00 00
[264917.814164] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814165] sd 6:0:0:0: [sdb]  
[264917.814165] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814166] sd 6:0:0:0: [sdb] CDB: 
[264917.814167] Write(10): 2a 00 09 1b a0 60 00 04 00 00
[264917.814175] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814177] sd 6:0:0:0: [sdb]  
[264917.814180] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814182] sd 6:0:0:0: [sdb] CDB: 
[264917.814183] Write(10): 2a 00 09 1b dc 70 00 04 00 00
[264917.814190] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814191] sd 6:0:0:0: [sdb]  
[264917.814192] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814193] sd 6:0:0:0: [sdb] CDB: 
[264917.814193] Write(10): 2a 00 09 1b 94 60 00 04 00 00
[264917.814286] sd 6:0:0:0: [sdb] Unhandled error code
[264917.814287] sd 6:0:0:0: [sdb]  
[264917.814288] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[264917.814289] sd 6:0:0:0: [sdb] CDB: 
[264917.814290] Write(10): 2a 00 09 1b e0 70 00 04 00 00
[264917.972038] sd 6:0:0:0: rejecting I/O to offline device
[264917.972310] Buffer I/O error on device dm-6, logical block 0
[264917.972562] lost page write due to I/O error on dm-6
[264917.972565] EXT4-fs error (device dm-6): __ext4_journal_start_sb:62: Detected aborted journal
[264917.972842] EXT4-fs (dm-6): Remounting filesystem read-only
[264917.973149] EXT4-fs (dm-6): previous I/O error to superblock detected
[264917.973459] sd 6:0:0:0: rejecting I/O to offline device
[264917.973767] Buffer I/O error on device dm-6, logical block 0
[264917.974084] lost page write due to I/O error on dm-6
[264942.917540]  connection1:0: detected conn error (1020)
[264991.186012] switch: port 2(tap0) entered disabled state
[264991.186248] device tap0 left promiscuous mode
[264991.186261] switch: port 2(tap0) entered disabled state

Best Resgards,
sluo