Bug 687403

Summary: VDSM Error during deactivateStorageDomain while diconnecting network between host and ISO domain
Product: Red Hat Enterprise Linux 6 Reporter: Rami Vaknin <rvaknin>
Component: vdsmAssignee: Saggi Mizrahi <smizrahi>
Status: CLOSED ERRATA QA Contact: Rami Vaknin <rvaknin>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1CC: abaron, bazulay, danken, iheim, mgoldboi, smizrahi, syeghiay, tdosek, yeylon, ykaul
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: vdsm-4.9-66 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 07:09:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
VDSM Logs none

Description Rami Vaknin 2011-03-15 15:02:21 UTC
Created attachment 485337 [details]
VDSM Logs

Scenario:
1. Configure NFS Data Center, connect few NFS DSD and an ISO domain on a different storage server
2. Disconnect network between host and ISO domain using an iptables rule

Thread-2924::DEBUG::2011-03-15 14:50:17,024::resourceManager::575::ResourceManager::(releaseResource) This is a shared lock. Granting all shared requests
Thread-3581::DEBUG::2011-03-15 14:50:17,024::task::491::TaskManager.Task::(_debug) Task 6230cee4-bf6b-4f52-b025-f7bbd9315fc8: ref 1 aborting False
Thread-3581::INFO::2011-03-15 14:50:17,025::sp::1092::Storage.StoragePool::(updateVM) spUUID=1cceaba4-2aec-45fe-b4f7-eac35299698d sdUUID=None
Thread-2924::ERROR::2011-03-15 14:50:17,026::dispatcher::106::Storage.Dispatcher.Protect::(run) list assignment index out of range
Thread-2924::ERROR::2011-03-15 14:50:17,027::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
    result = ctask.prepare(self.func, *args, **kwargs)
  File "/usr/share/vdsm/storage/task.py", line 1173, in prepare
    raise self.error
IndexError: list assignment index out of range



Thread-3737::DEBUG::2011-03-15 14:50:16,921::clientIF::42::vds::(wrapper) [10.35.64.205]::call getVdsCapabilities with () {}
Thread-2924::ERROR::2011-03-15 14:50:16,992::task::854::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 862, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/spm.py", line 114, in run
    return self.func(*args, **kwargs)
  File "/usr/share/vdsm/storage/spm.py", line 1168, in public_deactivateStorageDomain
    pool.deactivateSD(sdUUID, msdUUID, masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 984, in deactivateSD
    self.setDomains(sdUUID, sd.DOM_ATTACHED_STATUS, remove=False)
  File "/usr/share/vdsm/storage/sp.py", line 1357, in setDomains
    self.refresh()
  File "/usr/share/vdsm/storage/sp.py", line 1080, in refresh
    self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 1045, in __rebuild
    d = SDF.produce(dom)
  File "/usr/share/vdsm/storage/sdf.py", line 30, in produce
    newSD = cls.__sdc.lookup(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 98, in lookup
    self._refreshDomains()
  File "/usr/share/vdsm/storage/misc.py", line 1028, in helper
    return sm(*args, **kwargs)
  File "/usr/share/vdsm/storage/misc.py", line 1015, in __call__
    self.__lastResult = self.__func(*args, **kwargs)
  File "/usr/share/vdsm/storage/sdc.py", line 140, in _refreshDomains
    nfsSD.getFileStorageDomainList() +
  File "/usr/share/vdsm/storage/nfsSD.py", line 146, in getFileStorageDomainList
    dl.append(NfsStorageDomain(os.path.basename(os.path.dirname(f))))
  File "/usr/share/vdsm/storage/fileSD.py", line 70, in __init__
    for path in misc.tmap(checkPath, os.listdir(mountPath)):
  File "/usr/share/vdsm/storage/misc.py", line 1086, in tmap
    results[i] = result
IndexError: list assignment index out of range



Thread-238::DEBUG::2011-03-15 16:24:44,069::sdc::54::Storage.StorageDomainCache::(_refreshIfDirty) Cache is cold-expired
Thread-238::DEBUG::2011-03-15 16:24:44,070::misc::1011::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc._refreshDomains)
Thread-239::ERROR::2011-03-15 16:25:58,508::sp::84::Storage.StatsThread::(run) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/sp.py", line 78, in run
    self._domain = SDF.produce(self._sdUUID)
  File "/usr/share/vdsm/storage/sdf.py", line 30, in produce
    newSD = cls.__sdc.lookup(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 98, in lookup
    self._refreshDomains()
  File "/usr/share/vdsm/storage/misc.py", line 1028, in helper
    return sm(*args, **kwargs)
  File "/usr/share/vdsm/storage/misc.py", line 1015, in __call__
    self.__lastResult = self.__func(*args, **kwargs)
  File "/usr/share/vdsm/storage/sdc.py", line 140, in _refreshDomains
    nfsSD.getFileStorageDomainList() +
  File "/usr/share/vdsm/storage/nfsSD.py", line 146, in getFileStorageDomainList
    dl.append(NfsStorageDomain(os.path.basename(os.path.dirname(f))))
  File "/usr/share/vdsm/storage/fileSD.py", line 70, in __init__
    for path in misc.tmap(checkPath, os.listdir(mountPath)):
  File "/usr/share/vdsm/storage/misc.py", line 1086, in tmap
    results[i] = result
IndexError: list assignment index out of range

Comment 2 RHEL Program Management 2011-04-04 02:12:21 UTC
Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

Comment 3 Saggi Mizrahi 2011-04-27 08:49:39 UTC
should be fixed in 6.1

Comment 5 Saggi Mizrahi 2011-04-27 11:22:52 UTC
My bad, I thought is was in.
This is the patch
http://gerrit.usersys.redhat.com/#change,245

Comment 7 Tomas Dosek 2011-05-25 07:20:04 UTC
Verified - vdsm-4.9-67.el6 - IndexError no longer raises under above specified scenario. User is warned about error in rhevm.

Comment 8 errata-xmlrpc 2011-12-06 07:09:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html