Bug 1002049

Created attachment 791349 [details]
## Logs rhevm, vdsm, libvirt, thread dump, superVdsm

Description of problem:
Wrong behavior Host and Storage Domain during block connectivity to one of Storage Servers

Version-Release number of selected component (if applicable):
RHEVM 3.3 - IS11 environment:

RHEVM:  rhevm-3.3.0-0.16.master.el6ev.noarch
PythonSDK:  rhevm-sdk-python-3.3.0.11-1.el6ev.noarch
VDSM:  vdsm-4.12.0-72.git287bb7e.el6ev.x86_64
LIBVIRT:  libvirt-0.10.2-18.el6_4.9.x86_64
QEMU & KVM:  qemu-kvm-rhev-0.12.1.2-2.355.el6_4.5.x86_64
SANLOCK:  sanlock-2.8-1.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Crete iSCSI Data Center (DC) with two Hosts connected to 3 Storage Domain (SD)
* On first Storage Server (EMC) - one SD
* On second Storage Server (XIO) - two SD’s
see diagram below:
---------------------------------------------------------------------------
[V] Host_01 (SPM) _____ connected _________ SD_01 (EMC)
[V] Host_02 _______|                |_______ SD_02 (XIO)
                                    |_______ SD_03 (XIO) - Master
-------------------------------------------------------------------------------
2. From both Hosts block connectivity to SD_01 via iptables

Actual results:
Host_01 - Unassigned
Host_02 (SPM) - UP

SD_01 - Active
SD_02 - Active
SD_02 - Active

Expected results:
All Hosts UP
Blocked Storage Domain should become “Inactive”
Rest (unblocked) Storage Domain should be “UP”

Impact on user:
Failed work with this DC

Workaround:
none

Additional info:

r/log/ovirt-engine/engine.log

/var/log/vdsm/vdsm.log

Host_01
for i in `vdsClient -s 0 getStorageDomainsList`; do vdsClient -s 0 getStorageDomainInfo $i;  done
	uuid = eea45531-00c9-4585-b8f4-797fe948b6c2
	vguuid = xU6Y3n-y8ON-FgpO-0gJa-txCC-rj1N-x0EY4G
	lver = 7
	state = OK
	version = 3
	role = Master
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = 2
	type = ISCSI
	class = Data
	master_ver = 1
	name = SD-g-01


	uuid = bee4addd-78a2-4e67-8be7-e69230c282da
	vguuid = mYUphx-MM9z-A5Oz-d0QE-Reub-ydKm-TZgYfv
	lver = -1
	state = OK
	version = 3
	role = Regular
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = -1
	type = ISCSI
	class = Data
	master_ver = 0
	name = SD-g-02

	uuid = 7ccfcf85-bcf5-4219-9ccb-d9bb005dd195
	vguuid = Cbhs5s-hMe7-Ws3K-bWj2-7Gpv-9gBk-pUm3xX
	lver = -1
	state = OK
	version = 3
	role = Regular
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = -1
	type = ISCSI
	class = Data
	master_ver = 0
	name = SD-g-03


Host_02
for i in `vdsClient -s 0 getStorageDomainsList`; do vdsClient -s 0 getStorageDomainInfo $i;  done

	uuid = eea45531-00c9-4585-b8f4-797fe948b6c2
	vguuid = xU6Y3n-y8ON-FgpO-0gJa-txCC-rj1N-x0EY4G
	lver = 7
	state = OK
	version = 3
	role = Master
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = 2
	type = ISCSI
	class = Data
	master_ver = 1
	name = SD-g-01

	uuid = 7ccfcf85-bcf5-4219-9ccb-d9bb005dd195
	vguuid = Cbhs5s-hMe7-Ws3K-bWj2-7Gpv-9gBk-pUm3xX
	lver = -1
	state = OK
	version = 3
	role = Regular
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = -1
	type = ISCSI
	class = Data
	master_ver = 0
	name = SD-g-03

	uuid = bee4addd-78a2-4e67-8be7-e69230c282da
	vguuid = mYUphx-MM9z-A5Oz-d0QE-Reub-ydKm-TZgYfv
	lver = -1
	state = OK
	version = 3
	role = Regular
	pool = ['713c34f7-9407-4b41-ac9c-47176c50e8ea']
	spm_id = -1
	type = ISCSI
	class = Data
	master_ver = 0
	name = SD-g-02



	
2013-Aug-27, 20:25
	
Failed to connect Host tigris01.scl.lab.tlv.redhat.com to Storage Pool DC-g-01
	
	
2013-Aug-27, 20:24
	
Invalid status on Data Center DC-g-01. Setting status to Non Responsive.
	
	
2013-Aug-27, 20:24
	
Host tigris01.scl.lab.tlv.redhat.com cannot access one of the Storage Domains attached to the Data Center DC-g-01. Setting Host state to Non-Operational.
	
	
2013-Aug-27, 20:18
	
Storage Domain SD-g-02 (Data Center DC-g-01) was deactivated by system because it's not visible by any of the hosts.
	
	
2013-Aug-27, 20:18
	
Failed to Reconstruct Master Domain for Data Center DC-g-01.
	
	
2013-Aug-27, 20:16
	
Storage Domain SD-g-03 (Data Center DC-g-01) was deactivated by system because it's not visible by any of the hosts.
	
	
2013-Aug-27, 20:16
	
Invalid status on Data Center DC-g-01. Setting Data Center status to Non Responsive (On host tigris01.scl.lab.tlv.redhat.com, Error: Network error during communication with the Host.).
	
	
2013-Aug-27, 20:13
	
Storage domain SD-g-03 experienced a high latency of 125.299 seconds from host tigris02.scl.lab.tlv.redhat.com. This may cause performance and functional issues. Please consult your Storage Administrator.